Liquid immersion cooling vessel and components thereof

ABSTRACT

A two-phase liquid immersion cooling system is described in which heat generating computer components cause a dielectric fluid in its liquid phase to vaporize. The dielectric vapor is then condensed back into a liquid phase and used to cool the computer components. Using a pressure controlled vessel and pressure controller, the disclosed system may be operated at less than ambient pressure. By controlling the pressure at which the system operates, the user may influence the temperature at which the dielectric fluid vaporizes and thereby achieve increased performance from a given computer component. Utilizing robotic arms and slot-in computing components, a self-healing computing system may be created.

FIELD OF THE INVENTION

The present inventions are directed to liquid immersion cooled computingsystems, namely liquid immersion cooled computing systems utilizingpressure and/or vapor management.

SUMMARY OF THE INVENTION

Traditional computing and/or server systems utilize air to cool thevarious components. Traditional liquid or water cooled computers utilizea flowing liquid to draw heat from computer components but avoid directcontact between the computer components and the liquid itself. Thedevelopment of electrically non-conductive and/or dielectric fluidenables the use of immersion cooling in which computer components andother electronics may be submerged in a dielectric or electricallynon-conductive liquid in order to draw heat directly from the componentinto the liquid. Immersion cooling can be used to reduce the totalenergy needed to cool computer components and may also reduce the amountof space and equipment necessary for adequate cooling.

In disclosed embodiments of the invention described below, the use ofvapor and pressure management systems, as well as power managementsystems may be utilized, individually or in combination, to createsignificantly improved computer systems utilizing liquid immersioncooling.

Embodiments of the disclosed inventions relate to a pressure controlledvessel which may be used to house a liquid immersion cooled computingsystem. In some embodiments, the pressure controlled vessel contains asufficient quantity of liquid dielectric fluid to substantially immerseheat generating computer components and also contains an atmospherecomprising gaseous dielectric fluid. Embodiments further comprise acondensing system in order to cool and convert gaseous dielectric fluidto liquid dielectric fluid. The disclosed pressure management systemallows the disclosed embodiment to operate under a vacuum, therebyreducing the temperature at which dielectric fluid vaporizes and thecomputing system operates. Disclosed embodiments allow for increaseddensity of computer components and/or computing power due to theimproved temperature management system described.

BRIEF DESCRIPTION OF THE FIGURES

FIGS. 1-2 show a schematic of a pressure controlled vessel according toan example embodiment.

FIG. 3 shows the exterior of an exemplary embodiment of a pressurecontrolled vessel 110.

FIG. 4 depicts an exemplary embodiment of a super structure containingmultiple pressure controlled vessels.

FIG. 5 depicts an exemplary data center embodiment showing multiplepressure controlled vessels connected to a central power supply.

FIG. 6 depicts an exemplary data center embodiment showing multiplepressure controlled vessels connected to each other in series.

FIGS. 7A-D depict an exemplary embodiment of a cooled computing systemwith an interior robotic arm, airlock, and exterior robotic arm.

FIGS. 8A-C show an example embodiment of a rack system.

FIGS. 9A-G show an example embodiment of a chassis for mounting variouscomponents.

FIGS. 10A-F show an example embodiment of a pressure controlled vessel.

FIG. 11 shows an example cooling and vapor management system for apressure controlled vessel.

FIGS. 12A-E show another embodiment of the vessel.

FIG. 13 shows an example of a self-contained vessel.

FIG. 14 shows an example of an outer housing for the self-containedvessel.

FIGS. 15A-D show an example magazine located on a platform capable ofextending out of the vessel.

FIG. 16 shows a vapor recovery system according to an exemplaryembodiment.

FIG. 17 shows an exemplary embodiment of a rack power distributionsystem.

DETAILED DESCRIPTION OF THE INVENTION

In the following description, certain details are set forth such asspecific quantities, sizes, arrangements, configurations, components,etc., so as to provide a thorough understanding of the presentembodiments disclosed herein. However, it will be evident to those ofordinary skill in the art that the present disclosure may be practicedwithout such specific details. In many cases, details concerning suchconsiderations and the like have been omitted inasmuch as such detailsare not necessary to obtain a complete understanding of the presentdisclosure and are within the skills of persons of ordinary skill in therelevant art.

The equipment, components, systems, and subsystems of some disclosedembodiments below are described in terms of trade-names. It will beevident to those of ordinary skill in the art that the presentdisclosure may be practiced with many similar components whether or notsuch components are developed and/or sold under a particular trade nameand that the features and/or limitations associated with a particulartrade name components are not necessary to practice the disclosedinventions.

Dielectric Fluid

One aspect of immersion cooling is the use of a thermally conductive,but electrically substantially non-conductive or substantiallydielectric fluid. Examples of such fluids include some of the Novec™series of engineered fluids by 3M™ including Novec 7100, although thedescribed inventions are not limited to any particular dielectric fluid.Some immersion fluids typically have a boiling point at which it isdesirable to operate the cooled computer components. All computercomponents as well as other aspects of the disclosed systems arepreferably made of materials which are not soluble and do not otherwisebreakdown within the pressure controlled vessel when in contact with thedielectric fluid. In some embodiments, the boiling point of thedielectric fluid at standard atmospheric pressure may be less than about100° C., or less than about 80° C., or less than about 60° C., or lessthan about 50° C. or even lower. In some embodiments, the boiling pointof the dielectric fluid at standard atmospheric pressure is greater thanabout 60° C. or greater than about 40° C., or greater than about 30° C.or greater than about 20° C. Certain embodiments of immersion coolingfluids generally have a low vapor pressure. Some embodiments ofimmersion cooling fluids are fluorocarbons and/or fluorinated ketones.Certain embodiments of dielectric fluid may have a chemical formula of,or similar to, (CF3)2CFCF2OCH3, C4F9OCH3, or CF3CF2CF2CF2OCH3. Certaindielectric fluids comprise hydrofluoro ethers, methoxy-nonaflurobutane.

Other desirable characteristics of immersion cooling fluids include lowtoxicity, non-flammable, and/or low surface tension. In someembodiments, the immersion cooling fluid does not substantially harmcomputer components and/or the connections, wires, cables, seals and/oradhesives associated with computer components at the pressures andtemperatures utilized for liquid immersion cooling. Some dielectricfluids have a dielectric constant ranging from about 1.8 to about 8 anda dielectric strength of about 15 megavolts per meter (MV/m). In someembodiments, dielectric fluids have a dielectric strength of at leastabout 5 MV/m, or at least about 8 MV/m, or at least about 10 MV/m, or atleast about 12 MV/m. In some embodiments, dielectric fluids have adielectric strength of at most about 3 MV/m, or at most about 5 MV/m, orat most about 8 MV/m. In disclosed embodiments, any liquid in contactwith computer components 170 has a high enough dielectric strength toavoid damaging the computer components at the spacing and conditions ofthe specific application.

Some dielectric fluids have a critical heat flux of at least about 10W/cm2, or at least about 15 W/cm2, or at least about 18 W/cm2, or atleast about 20 W/cm2. Some dielectric fluids have a critical heat fluxof at most about 15 W/cm2, or at most about 10 W/cm2, or at most about 8W/cm2, or at most about 5 W/cm2.

FIG. 1 shows a schematic of a cooled computing system 110 according toan example embodiment. Embodiments of the disclosed cooled computingsystem 110 (or computing system, system, vessel, or pressure controlledvessel, all of which can be used interchangeably) may utilize a liquiddielectric fluid 140 to cool computer component 170 by immersing thecomponent into a bath of the fluid. As electricity is passed through thecomponent 170, the component 170 generates heat. As the component 170heats up, the performance of the component may be reduced or thecomponent may be damaged to the point of failure. It is advantageous tomaintain the various computing components at a stable and relatively lowtemperature. In some embodiments, computer component 170 may be kept atless than about 80° C., or less than about 70° C., or less than about65° C., or less than about 60° C., or less than about 55° C. In someembodiments, computer component 170 may be maintained at greater thanabout 60° C., or greater than about 50° C., or greater than about 40°C., or greater than about 35° C., or greater than about 30° C. As thecomputer component 170 heats up, heat is transferred to the liquiddielectric fluid 140 surrounding the component 170. When the liquiddielectric fluid reaches its boiling point, it will shift from a liquidphase into a gaseous phase and rise out of the liquid bath 142. Thecomponents 170 in the bath 142 of dielectric fluid may generally bemaintained at about the boiling point of the particular dielectric fluid140 being used.

When the liquid dielectric fluid is heated to the point of vaporizationat the pressure employed for a given application and becomes a gas,bubbles of the dielectric vapor will rise out of the liquid bath 142 andrise to the top of the system 110. The vapor is then cooled to be pointof condensing using condenser 130. Depending on the configuration of thesystem 110, the heating and cooling of dielectric fluid from liquidphase to vapor phase and back, can create a convection current as shownin FIG. 2.

In some embodiments, computer component 170 will be entirely submergedwithin liquid dielectric fluid 140 when the system is operating. Inother words, the upper portion of the computer component 170 is belowthe level of the dielectric liquid 140. It will be appreciated that asthe heat from computer components causes the dielectric fluid to changefrom liquid phase to gaseous phase, small bubbles of dielectric fluidvapor will be in contact with the computer components. Such componentswill still be considered entirely submerged within the liquid phase ofthe dielectric fluid. In some embodiments, the computer component 170may be submerged within the liquid phase of the dielectric fluid 140. Inone example embodiment, if any portion of a computer component,including but not limited to a motherboard, chip, server, card, blade,any portion of a GPU or CPU, and/or any peripheral component, is indirect contact with the liquid phase of the dielectric fluid 140, thecomputer component will be considered to be submerged. In certainembodiments, the computer component 170 may be at least partiallysubmerged within the liquid phase of the dielectric fluid 140. If thecomputer component 170 is not submerged, but is sufficiently cooled bydielectric vapor, the computer component will be considered to be atleast partially submerged.

In some existing immersion cooling systems, dielectric fluid must beconstantly added to the bath of dielectric fluid as the fluid isconsistently boiled off. Failure to add to the dielectric fluid to thebath 142 may result in the level of the dielectric fluid in the bath 142dropping until components are exposed to the gaseous atmosphere and notadequately cooled. This could result in decreased performance or damageto the component 170.

In some embodiments, there may be multiple operational modes which maybe accounted for with a fluid management system relating to thedielectric fluid in its liquid state. These modes may include, (1)Initial filling, which is the process by which dielectric fluid istransferred from a storage system into the vessel; (2) Continuousleveling, which is the process by which additional fluid is added, orexcess fluid is removed, to and from the vessel; (3) Unfilling, which isthe process by which the fluid is evacuated from the vessel and placedinto the storage system; and (4) Operational filtering, which is theprocess by which the fluid is continually cycled through a filteringsystem to ensure the removal of any particulates.

In some embodiments, the first three liquid management objectives, i.e.,initial filling, continuous leveling and unfilling, may be accomplishedthrough the same overall set of piping, pumps and valves. A dedicatedtank for storing liquid coolant may be used for the storage of new andexcess fluid which is removed and re-condensed during the vapormanagement process. A set of pipes and pumps may be used to bring thecoolant (or dielectric fluid) from the storage system to the vesselduring filling and leveling, and back out of the vessel and into thestorage system during unfilling operations.

In some embodiments, the fourth of the liquid management objectives, theoperational filtering, may be achieved through a series of skimmersand/or filters. The first stage may be a large particle filter locatedwithin the bottom of the vessel. The purpose of this filter is toprevent particles which are too large to be handled by the later stagesfrom entering the rest of the system. The second stage may be a mediumparticulate filter which sits in-line in the piping system between thefirst and third stage. This second stage medium particulate filter mayuse a small barrel style filter to remove particulates that were toosmall to be removed by the first stage filter but still too large to behandled by the third stage filter. The third stage filter may consist ofone or more parallel filters with support for various kinds of filterconfigurations. In some embodiments, the particular style of filter willbe dictated by conducting an analysis of the fluid after it has beenexposed and operating with a set of hardware components located withinthe vessel environment. Differing hardware and/or components are likelyto produce differing types of particulates and chemicals which may needto be filtered to ensure the long term life and efficiency of thedielectric fluid.

Pressure Management

In general, immersion cooling fluid must be kept free of dust, water,and/or other contamination. As the computer components 170 are in directcontact with the immersion cooling fluid 140, minor contaminants canresult in short circuits or damage to the computer components.Additionally, water or water vapor that may contaminate the dielectricfluid can reduce the dielectric properties, including, but not limitedto the dielectric strength, of the fluid as it becomes contaminated. Ifthe dielectric strength of the dielectric fluid is reduced, the computercomponents may short circuit or be otherwise damaged while in operation.One manner of reducing contamination is to operate an immersion coolingsystem in an enclosure which is kept at slightly higher or higher thanatmospheric pressure.

As the computer components 170 operate, the heat generated from theinitial use of the computer components causes some dielectric liquid 140to vaporize into a gas. If the immersion cooling system is confinedwithin a substantially enclosed housing, this vaporization typicallyincreases the pressure of the atmosphere within the housing. Pressurerelief valves, expanding enclosures, and/or other techniques may be usedto limit the increasing pressure and/or maintain the pressure within thehousing at or only slightly above atmospheric pressure. Maintaining aslight positive pressure in the enclosure may help to reduce theinfiltration of dust, water vapor, or other contaminants into theimmersion cooling computing system.

Current embodiments utilize an enclosed pressure controlled vessel 110(or cooled computing system 110) enclosure to contain the computingcomponent 170 and immersion cooling equipment, as well as the associatedpower supplies, networking connects, wiring connections, and the likewithin a pressure controlled vessel. In contrast to existing models, thepressure controlled vessel 110 may be maintained at least at a slightvacuum, thereby reducing the boiling point of the dielectric fluid 140to a temperature below its boiling point at standard atmosphericpressure.

By operating the computing and immersion cooling system under a vacuum,the components 170 may be maintained at the reduced, low-pressureboiling point of the dielectric fluid 140. This has the benefit ofincreased cooling which allows for more electricity to be passed throughthe various components 170 resulting in greater performance of thecomponents. By controlling the pressure in the pressure controlledvessel 110, the boiling point of the dielectric fluid 140 may also becontrolled, thereby allowing the same fluid 140 to be used in a broaderrange of conditions. Many embodiments benefit from cooler temperatures,however certain computer components 170 have an ideal range and sufferfrom reduced performance at temperatures below that range. Bycontrolling the pressure in the pressure controlled vessel 110, theboiling point of the immersion cooling fluid 140 may also be controlled.In certain embodiment, the disclosed pressure management system may beused to dynamically control the pressure, and thereby the boiling pointof the dielectric fluid 140 as the computing system is initiated, shutdown, or in response to other changing conditions.

In addition to reducing the boiling point of the dielectric fluid 140 byoperating in a pressure controlled vessel 110 at less than ambientpressure, a computer component 170 itself may be modified in order tomore efficiently transfer heat away from itself and into the dielectricfluid 140. By increasing the surface area of a component 170, forexample, a chip, which is exposed to the liquid dielectric fluid 140,heat transfer between the component 170 and the bath 142 of dielectricfluid 140 may be increased. An exemplary device for increasing surfacearea may be a copper boiler or a copper disc, which may be adhered to achip of other computer component 170. In certain embodiments, theadhesive used will be selected based on its ability to transfer heat andits solubility in the dielectric cooling fluid. Preferred adhesivesexhibit high thermal conductivity and low solubility in the selecteddielectric fluid.

FIG. 1 shows a schematic of an exemplary embodiment of the disclosedcomputing system. Embodiments of the disclosed systems include apressure controlled vessel 110 (or the cooled computing system 110), apressure controller 150, an immersion cooling system comprising at leasta volume of dielectric fluid 140 and a condensing structure 130, and thedesired computer components 170. A pressure system may be configured tomaintain the desired degree of reduced pressure. The pressure controlledvessel 110 may be configured to maintain a negative pressure while stillallowing multiple penetrations into the pressure controlled vessel 110for various connections including, but not limited to power, data,networking, cooling water, and/or communications systems. Someembodiments utilize hermetic and/or marine grade connections. Operatinga computer system within a pressure controlled vessel 110 at less thanambient pressure requires a series of modifications to the system as awhole. These modifications are discussed below and some are readilyapparent to one of ordinary skill in the art.

FIG. 3 shows the exterior of an exemplary embodiment of a pressurecontrolled vessel 110. In some embodiments, the disclosed pressurecontrolled vessel 110 is at least about 2 feet tall, or at least about 3feet tall, or at least about 4 feet tall, or at least about 5 feet tall.In some embodiments, the pressure controlled vessel is at most about 3feet tall, or at most about 4 feet tall, or at most about 5 feet tall.

In certain embodiments, the pressure controlled vessel has an interiorvolume of at least about 100 cubic feet, or at least about 150 cubicfeet, or at least about 200 cubic feet, or at least about 250 cubicfeet, or at least about 300 cubic feet, or at least about 350 cubicfeet, or at least about 400 cubic feet.

In some embodiments, the pressure controlled vessel will be configuredto contain about 12 vertical inches of liquid dielectric fluid and about36 vertical inches of dielectric fluid vapor while in operation. Incertain embodiments, the ratio of liquid volume to gaseous volume helpsto create a convective current and direct gaseous dielectric vaportowards condensing structures which turn the vapor back into a liquid.In some embodiments, the pressure controlled vessel is configured tocontain a ratio of a volume of liquid dielectric fluid to a volume ofgaseous dielectric fluid of about 1:6 during operation. In otherembodiments, the pressure controlled vessel is configured to contain aratio of a volume of liquid dielectric fluid to a volume of gaseousdielectric fluid of about 1:3, or about 1:5, or about 1:8 or about 1:10,or about 1:15 during operation.

In one example embodiment, the pressure management system may include apressure controller 150. The pressure controller 150 can be a source ofvacuum, e.g., the pressure controller 150 may be a vacuum pump which maybe connected to the pressure controlled vessel 110. In some embodiments,the vacuum pump 150 may be remote and the vacuum may be transmitted tothe pressure controlled vessel 110 using piping and/or tubing. Inpreferred embodiments, a pressure sensor 180 is contained within thepressure controlled vessel 110 and used to regulate and/or maintain thedesired negative pressure within the pressure controlled vessel 110. Insome embodiments, the pressure sensor 180 and/or a pressure regulator190 may be connected to a processor which monitors the pressure in thepressure controlled vessel 110 using the pressure sensor 180 andregulates the pressure using the pressure regulator 190.

Some embodiments comprise operator protection mechanisms. In one exampleembodiment, the operator protection mechanism may be a locking mechanismthat precludes the system from operating if any of the lids or servicepanels to the pressure controlled vessel are not in place. In oneexample embodiment, the operator protection mechanism may include acontroller to immediately power down the system in the event of anunauthorized breach of one of the doors or panels of the pressurecontrolled vessel. In addition to providing a life safety feature, theoperator protection mechanism may also provide an enhanced operationssecurity feature for deployments where sensitive data is housed withinthe vessel. By ensuring that the equipment cannot be accessed duringnormal operation without shutting down power to the system, a high levelof assurance can be achieved in the efficiency of disk protectionmechanisms. Furthermore, in some embodiments, the disk protectionmechanisms may use runtime stored encryption keys to protect data atrest on the pressure controlled vessel.

In certain embodiments, in addition to denying unsafe access to thepressure controlled vessel, sensors may be placed to ensure that thesystem is operating as designed. The primary sensor package may includetemperature sensors in the vapor space; temperature sensors in theliquid space; humidity sensors in the vapor space; and/or pressuresensors in the vapor space. These sensor readings may be monitored bysoftware and/or by human operators to ensure that the system isoperating in a safe and correct fashion. In some embodiments, the sensordata will be recorded or later analyzing.

In some embodiments, additional sensors may be incorporated within thevessel or the super structure (defined below). Such sensors couldinclude, for example, FLIR based heat imaging cameras; VESDA or otherforms of aspirating smoke detectors; and/or refrigerant leak detectorsdesigned to detect a leak of the dielectric fluid into the surroundingenvironment.

In some embodiments, the vessel and/or super structure may be equippedwith indicator lights relating to the operational status of the system.

Although the cooled computing system 110 is sometimes referred to as thepressure controlled system 110, one or ordinary skill in the artrecognizes that many, if not all, of the benefits of the cooledcomputing system 110 can be realized without using a “pressurecontrolled system.”

Vapor Management System

Liquid immersion cooling systems may be operated in different ways. Somemay operate by continuously cooling the immersion fluid directly. Othersmay operate by allowing the liquid to reach its maximum liquid phasetemperature and then boil into a vapor phase. Immersion cooling systemswhich operate by allowing the liquid to evaporate are called two-phaseimmersion cooling systems. Two-phase immersion cooling systems oftenallow the dielectric fluid to boil and/or vaporize and regularly addadditional fluid to replace the fluid which is lost to the atmosphere.

Disclosed embodiments utilize a liquid immersion cooling system which iscontained within a pressure controlled vessel 110. This has theadvantage of not losing the dielectric fluid 140 even after it hasconverted to a gaseous form. In a closed, or substantially closedpressure controlled vessel 110, the gaseous dielectric fluid may becondensed and added back to the bath 142 of the liquid dielectric fluid140 which is actively used to cool the computing components 170. Thecondensing step may be performed in any convenient manner, for example,by running process water through a thermally conductive tube. Condensingstructures 130 may include radiator fins and/or similar equipment whichincreases the surface area of the condenser, thereby allowing greaterand/or more rapid condensation of the gaseous dielectric fluid andreturning it to a liquid form. In some embodiments the process water isat ambient temperature and is not actively cooled. In other embodiments,the process water may be chilled using evaporative cooling, dry coolingtowers, and/or other method of chilling process water known in the art.

In some embodiments, there may be two interfaces between a pressurecontrolled vessel and external systems. The first may be the processwater supply interface. This may be a pipe which delivers process waterfrom a facility which provides chilled process water to a distributionmanifold on the pressure controlled vessel. The second may be theprocess water return interface. This may be a pipe which returns theprocess water to the facility which provides chilled water. The processwater may be returned to the facility after the process water has flowedthrough the pressure controlled vessel and associated coolingcomponents. Cooling components may include, for example, condensers,condensing coils, and/or radiators within the vessel as well as coilswhich reject heat from the exhaust of any powered components including,for example, motors, pumps, and/or utility cabinets. In someembodiments, there may be two interfaces between a super structure andexternal systems. The interfaces may be similar or substantially similarto the two interfaces between the pressure controlled vessel andexternal systems.

In some embodiments, the location of the condensing structures 130within the pressure controlled vessel 110 may be configured in order tooptimize the flow of vapor phase dielectric fluid and increase the rateand/or efficiency of condensation. In some embodiments, the geometry ofthe pressure controlled vessel 110 itself may be controlled in order toincrease the rate and/or efficiency of condensation.

As shown in FIGS. 1-3, in one exemplary embodiment, a pressurecontrolled vessel is about 10 feet long, about 4 feet wide, and about 4feet tall. A bath 142 may be created within the pressure controlledvessel 110 using about 130 gallons of Novec™ dielectric fluid 140. Thisleaves a layer of liquid dielectric fluid about 12 inches deep in animmersion cooling tank at the bottom of the pressure controlled vessel,while the majority of the pressure controlled vessel volume is gaseous.The ceiling of the pressure controlled vessel is lower in the middle ofthe structure running lengthwise. The ceiling and/or lid 120 anglesupward and raises as it approaches the sidewalls of the pressurecontrolled vessel 110. Condensing structures 130 run lengthwise on twosides of the pressure controlled vessel 110. The condensing structures130 in this exemplary embodiment may be about 12 inches wide and about24 inches tall and run substantially the entire length of the pressurecontrolled vessel 110. The condensing structures 130 include radiatorlike material with high surface area fins which are cooled using flowingprocess water. Some embodiments may additionally or alternativelycomprise a heat exchanger.

As shown in FIG. 2, the structural arrangement within the pressurecontrolled vessel 110 directs a convective flow of dielectric fluidvapor as it rises from the liquid bath 142 after boiling. The structuralarrangement directs the convective flow up towards the ceiling of thepressure controlled vessel where the flow is directed toward the highsurface-area of condensing structures 130 and condensed back into aliquid form. The dielectric fluid 140 then flows back into the liquidbath 142. In this manner, the total amount of dielectric fluid 140 maybe conserved within this closed housing. The use of convective currentto circulate dielectric fluid vapor allows disclosed embodiments tooperate in the absence of a mechanical pump for circulating thedielectric liquid, thereby reducing the total energy usage of thedisclosed system.

Certain embodiments may utilize additional tanks and/or storagecontainers of dielectric fluid which may be used during star-up and/orshut-down of the system, in the event the pressure controlled vesselmust be opened, and/or to allow redundant and robust control of thelevel of liquid dielectric fluid.

FIG. 11 shows an example cooling and vapor management system 600 for apressure controlled vessel 110. In this example embodiment, the coolingand vapor management system 600 can include a chilled process waterstorage 611, which runs through the cooling coil 132 to causecondensation of the dielectric fluid 140. After passing through thecooling coil 132, the process water can proceed to a process waterreturn storage 612. The cooling and vapor management system 600 can alsoinclude a tank for vapor storage 614 and a tank for dielectric fluidstorage 615. The tanks 614 and 615 can provide dielectric fluid or vaporwhen needed, e.g., during star-up and/or shut-down of the system. In oneexample embodiment, the tanks 614 and 615 can be coupled via acondensing structure 616. In case there is an excess supply of vapor inthe tank 614, the condensing structure 616 can remove the vapor and addit as dielectric fluid to the fluid storage tank 615.

In some embodiments, during operation, the pressure controlled vessel ismaintained at about 3 psi less than ambient atmospheric pressure whichhelps to reduce the boiling point of the dielectric fluid and therebyreduce the operating temperature of the computer chips and othercomponents. In some embodiments, the pressure controlled vessel 110 ismaintained at least at about 2 psi below ambient pressure or at leastabout 4 psi, or at least about 6 psi, or at least about 8 psi, or atleast about 10 psi below ambient pressure.

In some embodiments, it will be necessary to select components with somedegree of tolerance for pressure fluctuations. It would be preferable,to use components which can withstand a wide degree of pressures toallow for manipulation of the coolant boiling point, and as such thegeneral operating temperature of the overall system, by adjusting theoperating pressure of the system. Given the operating nature of thetwo-phase system, standard operating conditions for some embodimentswould see a variance of between ±4 PSIg. In certain conditions, such asduring a rapid startup or shutdown of the system, a difference of threeadditional PSIg may be experienced. In some embodiments, system leveladjustments can be made to better control these variables and keep themwithin a more controlled and defined range.

In certain embodiments, the computer components 170 are operated atleast at about 3% less than ambient pressure, or at least about 5%, orat least about 10%, or at least about 15%, or at least about 20%, or atleast about 25%, or at least about 30% less than ambient pressure.

In some embodiments, the pressure controlled vessel is maintained,during operation at less than about 750 torr, or at less than about 710torr, or less than about 650 torr, or less than about 600 torr, or lessthan about 550 torr, or less than about 500 torr, or less than about 450torr, or less than about 400 torr, or lower. In some embodiments, thepressure controlled vessel is maintained, during operation at greaterthan about 650 torr, or greater than about 600 torr, or greater thanabout 550 torr, or greater than about 500 torr, or greater than about450 torr, or greater than about 400 torr, or greater than about 300torr.

Some embodiments utilize a vapor scrubbing process and/or initialpurging process in order to control the gaseous atmosphere within apressure controlled vessel. This process removes a portion of thegaseous atmosphere from the pressure controlled vessel and removesundesirable portions of the atmosphere such as air and water vapor.These, and other non-desirable portions of the atmosphere may beseparated based on the temperature at which the vapor condenses into aliquid. Due to the specialized nature and boiling point of thedielectric fluid, many naturally occurring contaminants may be removedusing this method. Removing the non-readily condensable fluids helps tomaintain the purity of the dielectric fluid. A fluid will be consideredto be not readily condensable if the condensation point of the fluid isgreater than about 20° C. lower than the condensation point of thedielectric fluid at standard atmospheric pressure or if the condensationpoint of the fluid is less than 10° C. at standard atmospheric pressure.

During maintenance, startup and/or shut down operations, a blanket ofinert gas, such as nitrogen, gas may be introduced into the pressurecontrolled vessel in order to reduce the amount of dielectric fluid lostwhen the pressure controlled vessel is opened and/or exposed toatmospheric conditions. As shown in FIG. 11, the cooling and vapormanagement system 600 can include an inert gas tank 613, which cansupply inert gas to reduce dielectric fluid loss.

Some disclosed embodiments may include a substantially self-containedserver and/or computing system. In some embodiments, specialized sealsand/or connections may be utilized to reduce the total number ofpenetrations into the pressure controlled vessel 110. Some embodimentscombine power, water, vacuum, and networking connections into a bundleof lines in order to minimize the penetrations into the pressurecontrolled vessel in order to reduce the potential for leaks while thesystem is under vacuum.

FIG. 4 depicts an exemplary embodiment of a super structure containingmultiple pressure controlled vessels. In this example embodiment, twopressure controlled vessels 110 are pre-plumbed, pre-wired and housedwithin a modular super structure 210. This allows for embodiments to bepre-fabricated and delivered as substantially complete, self-containedsystems. The modular system may be configured to be connected to othermodular embodiments of the disclosed computing system. In someembodiments, the modular super structure 210 will require only a singlepower connection and will be pre-wired with the appropriate electronicsto supply the required voltages to the computer components and/or otherelectronic components.

FIG. 5 depicts an exemplary data center embodiment showing multiplepressure controlled vessels connected to a central power supply. FIG. 6depicts an exemplary data center embodiment showing multiple pressurecontrolled vessels connected to each other in series. In these exampleembodiments, the pressure controlled vessels 110 may or may not beplaced within a superstructure.

FIGS. 7A-D depict an exemplary embodiment of a cooled computing systemwith an interior robotic arm, airlock, and exterior robotic arm. In thisexample embodiment, an internal robotic arm 230 contained within thepressure controlled vessel 110 may be used to remove a component 170 anddeliver the removed component to an airlock 220. Using the airlock 220,the component 170 may be removed without substantially disturbing ordisrupting the pressure, atmosphere, dielectric fluid, and/or the otherconditions within the pressure controlled vessel 110. Once the component170 is removed from the pressure controlled vessel 110, a replacementcomponent may be introduced into the pressure controlled vessel 110using the airlock 220. The replacement component may then be installedby the internal robotic arm 230. The use of components which may beinstalled in a “slot-in” manner, such as a blade server and chassis, mayfacilitate this process significantly.

A disruption to a condition within the pressure controlled vessel may bedetected by a sensor, e.g., pressure sensor, placed within the pressurecontrolled vessel. The disruption may be indicated by at least a 10%deviation in that condition outside of the standard range of operatingconditions. A significant disruption to a condition within the pressurecontrolled vessel may be indicated by at least a 30% deviation in thatcondition outside of the standard range of operating conditions.

In certain embodiments, a self-contained diagnostic program may runwhich analyzes the performance of the components within the pressurecontrolled vessel 110. If a component 170 is not performing as desired,a robotic arm 230 may be used to remove and/or replace the componentautomatically. In this manner a self-healing, self-contained serverand/or computing system may be created. In certain embodiments, such aself-healing system may be pre-fabricated and pre-wired to create amodular unit which may be shipped or delivered to remote locations usingconventional methods to provide significant high-efficiency computingpower which requires limited set-up and/or maintenance.

In some embodiments, a first vapor management objective of cooling thevapor and causing it to condense from a gas state back to a liquid stateis achieved entirely within the closed system of the vessel through theuse of condensing coils. Process water will be piped through condensingcoils within the vessel. The shape and geometry of the vessel itselfwill encourage the flow of vapor from the bath area to the coil area andgravity will serve to pull the re-condensed liquid back into the batharea.

In some embodiments, a second vapor management objective of monitoringand maintaining the internal pressure of the vessel is achieved throughthe use of integrated pressure sensors within the vessel and use of apurge system. In some embodiments, the purge system will be used toremove excess vapor from the vessel and condense it back to a liquid forstorage in the liquid storage tank.

In some embodiments, a third vapor management objective of controllingand removing non-condensable components of the vapor which are presentduring system startup is accomplished via the same mechanism as thesecond objective. The purge system may be used to bring the system underpressure during its initial startup and to remove any non-condensablegases from the system.

In some embodiments, a fourth vapor management objective of controllingthe overlay of an inert gas may be accomplished using a dedicatednitrogen overlay feeding system. This overlay keeps the coolant belowthe top of the vessel, allowing for minimization of the loss of coolantduring periods where the vessel is opened to service the componentstherein. Dedicated piping from a set of nitrogen storage tanks through aset of dedicated overlay pipes within the vessel will allow for theadding of the inert overlay when the operator desires to open thesystem. This gas, along with any other non-condensables, may be removedduring the non-condensable removal process which may occur at systemstartup. The overall vapor management process may be managed andmonitored through the control system software based on user commands andsystem state monitoring.

Ballast Blocks

In some embodiments of the disclosed system, such as that shown in FIG.1, the pressure controlled vessel 110 may include a deeper bath portion142 for containing the majority of the dielectric fluid 140 and abroader shelf area 112 adjacent to the bath. The boards, cards, chips,blades, and/or any other computer components 170 are substantiallycontained within the deeper bath section 142 of the pressure controlledvessel 110. The broader shelf area 112 may also contain liquiddielectric fluid 140 and/or collect dielectric fluid 140 that isre-condensed into the liquid phrase from the vapor phase. In certainembodiments, the depth of the dielectric liquid in the pressurecontrolled vessel 110 may be increased utilizing ballast blocks 160.Ballast blocks 160 may be used to occupy undesired volume on the shelf,thereby displacing any dielectric fluid 140 that would be on the shelf112 and raising the level of liquid without requiring the addition ofadditional dielectric liquid 140. In some embodiments, the ballastblocks 160 include riser feet 161 which allow fluid to flow underneaththe ballast blocks 160 so that condensed liquid can continue to flowinto the deeper bath portion of the pressure controlled vessel withoutthe flow being hindered by the ballast blocks 160.

The ballast blocks 160 may be made of any material that does notinterfere with the operation of the disclosed immersion cooling system.The ballast blocks may be made of materials including, but not limitedto, metals, rubbers, silicone, and/or polymers. Preferred materials arenot substantially soluble in the dielectric fluid. The blocks must bedenser than the dielectric fluid but are not required to be solid. Inpreferred embodiments, the blocks will have a handle or cut out whichallows the block to be more easily handled and manipulated. Someembodiments of the ballast blocks 160 utilize interlocking top andbottom sections so that the blocks maybe stacked on top of each other ina secure manner. The interlocking top and bottom reduce the risk of ablock damaging any nearby component if it slides or is otherwisedisplaced from its intended position. In some embodiments, theinterlocking top includes recessed portions which align with feet and/orrisers on the bottom portion such that the lowest block does not preventfluid flow and blocks may be securely stacked on top of the lowest blockin order to occupy a significant volume, thereby allowing the level ofdielectric liquid to be raised without requiring a significant amount ofadditional dielectric liquid to be added.

In some embodiments, the ballast blocks 160 are configured to run theentire length of the pressure controlled vessel 110 and/or shelf 112. Inother embodiments, the ballast blocks 160 may be substantially any sizewhich allows for the block to be handled. In such embodiments, multiplemodular ballast blocks may be configured to displace as large or assmall of a volume as desired. In some embodiments, a single ballastblock has an outer dimensions of about 2 feet long or about 3 feet longor about 4 feet long or longer and about 6 inches wide, or about 8inches wide, or about 12 inches wide, or wider, and about 1 inch tall,or about 3 inches tall, or about 6 inches tall, or about 8 inches tallor taller.

The Super Structure

The disclosed computing system consists of various components, all ofwhich may be attached, directly or indirectly to a physical superstructure 210, as shown in FIG. 4. The super structure 210 allows forpre-wiring and pre-plumbing of any required electrical, sensor, control,power, fluid control, pressure control, and/or communication systems.This allows for faster and simplified deployment in the field andtesting at the factory prior to delivery to the customer.

The super structure 210 is typically fabricated from metal componentsand may be skid mounted or configured to be handled with a forklift,hoist, or crane. In some embodiments, the super structure 210 isconfigured to fit within a standard container in order to facilitateshipping. The super structure 210 and associated components may beconfigured to weigh less than about 58,000 lbs total and may be dividedinto smaller subcomponents in order to facilitate shipping withoutrequiring special equipment. In some embodiments, super structure 210and the associated components will weigh less than about 50,000 lbs, orless than about 40,000 lbs, or less than about 30,000 lbs, or less thanabout 20,000 lbs. In some embodiments, super structure 210 and theassociated components will weigh more than about 5,000 lbs, or more thanabout 10,000 lbs, or more than about 20,000 lbs, or more than about30,000 lbs. Embodiments of the super structure 210 may be any size andor shape. Many embodiments are sufficiently large to contain multiplepressure controlled vessels 110, server racks 310, and the associatedliquid immersion cooling equipment as well as the necessary equipmentfor managing power delivery and distribution and network connectivity.

The overall design of the super structure 210 can be adjusted toaccommodate the unique aspects of each deployment, includingcustomizations to the types and quantities of power and process waterinterconnects to meet the needs of existing facilities.

The control and management systems for all of the components within thedisclosed pressure controlled vessel may be included as part of thedisclosed computing system. A preferred embodiment of the disclosedsystem includes all of the required mechanical systems to maintain andoperate a two-phase liquid immersion cooling environment, including therequired pumps, valves, regulators, vapor management systems, pressuremanagement systems, and other associated components.

The super structure 210 may be an open frame design, or may include sidepanels and access doors. This allows for deployment within existingstructures or outside in field locations. The super structure 210 may bemodified to include weatherization features, allowing for deployment inharsh environments. In some embodiments, the super structure may be askid/module framework.

Various systems, features and/or capabilities may be incorporated intothe super structure 210 to support, monitor, and manage the othercomponents of the pressure controlled vessel and any environmentscontained within or associated with the pressure controlled vessel. Insome embodiments, such systems may include fire detection and/orsuppression capabilities, dedicated air condition and/or environmentalmanagement capabilities, security features such as access control,and/or surveillance features among many others.

The Power System

Some embodiments of the super structure 210 are designed to acceptvarious means of electrical inputs and connect them to an existing powerdistribution system built within the super structure. One of manyexemplary embodiments includes a 415V input to a main breaker, which isthen distributed to a series of power shelves which converts the 415V ACinput to 12V DC output. In preferred embodiments, this conversion occursin substantially one conversion step, thereby reducing the lostefficiency typically associated with such conversions. Traditionalcomputer server locations typically convert incoming industrialelectricity from a high AC voltage, such as 415V to a reduced AC voltagesuch as 120V. This conversion results in a loss of energy to heat. Undercommon circumstances, this may result in about a 6% loss of energy.Then, the 120V electricity must be further converted to DC current forthe use by various computer component. This second conversion results ina second, about 6% loss of energy to heat. By directly convertingindustrial electricity of about 415V to about 12V DC, the total loss ofenergy to heat can be reduced

Another exemplary implementation may include the connection of a 480V ACinput to a power shelf which converts the 480V AC input to 48V DCoutput, which is then distributed to a series of intermediary powersupplies which converts the 48V DC input to various DC outputs,including, for example, 12V, 5V, 3.5V 3.3V and others.

On some implementations, there may be a single set of power supplies, orthere may be multiple power supplies operating at different input andoutput voltages. The exact configuration will be adjusted to meet theneeds of the particular equipment being installed and depending on theconditions of the application. The particular design of a power systemmay be adjusted to meet the needs of the particular environment in whichthe disclosed computing system is being deployed. Customizations mayinclude the type, capacity, and interfaces for both input and output ofpower to the system.

In some embodiments, a rack power distribution system may comprise amodular power supply system and/or set of modular power supply systems.The specific configuration of the modular power system or systems is notparticularly critical so long as it can deliver the desired quantity andtype of power to the rack. Accordingly, modular power systems may beconfigured in parallel or in series or in a combination thereof toprovide one or two or even multiple power distribution pathways. Thespecific pathways to the rack may be direct or indirect and often maydepend upon the components involved, power quantity and type, and/ordesired configuration. If desired, the pathway to the rack may involvedistributing power to a chassis located within the rack. The powerdistributed may be delivered at one or more desirable voltages which mayvary depending upon the configuration and components. In some casesdesirable voltages may include, for example, 12V, 5V and/or 3.5V. Insome embodiments, if a chassis is employed, then it may employ one ormore subsystems. Such subsystems may include any desired subsystem thatdoes not interfere with the desired quantity and type of power to bedelivered to the rack. As but one example a power-on-package subsystemmay be employed. Such a package may accept AC current and convert to DCcurrent and/or vice versa depending on what is desired. For example, aparticularly useful power-on-package subsystem may be designed to acceptinput power at 208, 240, 380, 400, 415, 480, and/or 600 volts AC andconvert that power directly to DC power, e.g., 48V DC.

The modular power supply system or systems may be powered directly orindirectly in any convenient manner. For example, a modular power supplysystem may be powered directly via a primary electrical distributionsystem within the chassis. Depending on the type and quantity of powerand other components, a chassis could use an interface, such as a set ofspring loaded pins or other suitable connector interface to establishelectrical continuity between the power distribution pathway and thechassis itself. Continuity could then be established between thatinterface connector and any desired power supply input interface ondesired servers or other computing components located on the chassis. Insome embodiments, a power-on-package module may be utilized within eachchassis to convert the voltage to the appropriate levels directly at thechassis itself. This could be used for various types of powerdistribution but may be especially useful with, for example, 48Vdistribution. FIG. 17 shows an exemplary embodiment of a rack powerdistribution system 950. In this example embodiment, a rack 310 canreceive an AC input 960 at an AC interface 311 of the rack 310. Thepower distribution system 950 can generate a DC output 320 anddistribute the DC output 320 to one or more chassis 400.

In some embodiments, ensuring supply of reliable power to the computercomponents within a rack is a primary concern. To that end, someembodiments use blade level power supplies or computer component levelpower supplies which may supply a certain input voltage and provide therequired output voltage to the blade and/or component level powersupplies. Some embodiments incorporate multiple power supplies into eachblade to provide redundancy.

In some embodiments, one or more switches may require power. Anexemplary switch may be a standard datacenter grade switch with theappropriate interfaces to connect to the backplane and provide racklevel communications to each blade. Some embodiments distribute only asingle voltage, and this can be accomplished by a power rail andinterface system with connectors to serve as the interface between apower rail and each of the blades, delivering the voltage eitherdirectly to the power supply input rails or via an intermediaryconnector to sit in between the power supply power leads and a racklevel voltage distribution system.

In some embodiments, there may be one or more power rails whichdistribute the primary voltage along the bottom of the rack. This railmay be fed from one or more primary power rectifiers, likely locatedoutside of the pressure controlled vessel and delivered to each rack viaa cable or busbar system. The use of higher voltage, for example, 48Volts, at this level may reduce the required current carrying capacityof the distribution system and may effectively interface between thedistribution rail(s) and the load interface.

In some embodiments, there will be two primary power distributionsystems located within the super structure platform. The first is thePrimary Equipment Power System (PEPS) and the second is the SecondaryEquipment Power System (SEPS). The purpose of the PEPS is to provideelectrical service to the components within the vessel. This system maybe a high voltage, high current distribution system which accepts inputsvia either copper conductors or busbar systems and delivers it to theprimary power supplies responsible for providing operational current tothe chassis, computer components and/or other critical load equipment.The power will enter the super structure at a defined point and beterminated to a master service disconnect breaker. Upstream of thispoint will be the electrical service and all power redundancy componentsused in the system. This input will be at a high voltage, such as, forexample, 415 or 480 volts AC. The primary equipment load will be drivenby power supplies or rectifiers which are fed from a breaker paneldownstream of the master disconnect breaker.

The purpose of the SEPS is to provide electrical service to all of theinfrastructure support systems and components located within the superstructure. As the components required as part of the secondary equipmentinfrastructure may expect a lower input voltage, the SEPS may be poweredby a step-down transformer which is connected upstream of the PEPSmaster service disconnect breaker via a secondary service disconnect.

This arrangement will allow for the super structure support andinfrastructure systems, including all of those components which arepowered from the SEPS, to be turned on and operate even without primarypower being delivered to the remainder of the system components. Allaspects of the management and control systems, as well as the vaporcontrol system, may be able to operate independently of the operation ofthe PEPS.

In some embodiments, an uninterruptable power supply (UPS) is includedas part of or in addition to the power distribution system. Theincorporation of the UPS allows for continued operation of the disclosedcomputing system in the event of a temporary interruption to theexternal power supply.

Components of the disclosed power distribution system may include, butis not limited to, commercially available components such as, forexample, uninterruptible power supplies, DC power systems, AC powersystems, and/or power control and monitoring systems. Some suchcomponents may include, but are not limited to Vertiv products such as,for example, Liebert and/or Chloride UPS products, dual conversiononline UPS, line-interactive UPS, stand-by UPS, lithium-ion battery UPSand combinations thereof. The UPS products may be single phase or threephase. Other exemplary power distribution system components may include,for example, Emerson Network Power products, NetSure DC power systems,Vertiv, Liebert, Chloride, and/or NetSure power distribution units andrelated components, such as, for example, inverters, rectifiers,transfer switches, and combinations thereof. Commercially availablemonitoring units, controller units, and/or software related to suchcomponents may also be incorporated into certain disclosed embodiments.

The Pressure Controlled Vessel and Pressure Management Systems

Embodiments of the disclosed system include a pressure controlled vesselwhich is designed to house a two-phase liquid immersion cooling system.The pressure controlled vessel 110 contains a bath 142 of dielectriccooling fluid 140, condenser 130 with cooling coils 132 to condensegaseous phase dielectric fluid into a liquid, and the physicalmechanisms and/or equipment necessary to hold computer components 170and distribute power from the power system to the equipment andcomponents located within the pressure controlled vessel 110.

During operation, the pressure controlled vessel 110 may be kept at aslight vacuum. It will be appreciated that a variety of specializedconnections and considerations must be made in order to operate acomputing system within a pressure controlled vessel 110 which ismaintained at a negative pressure.

Some embodiments of the disclosed system use a series of fiber opticMedia Transfer Protocol (MTP) interfaces allowing connectivity of fiberinto the pressure controlled vessel 110 in addition to break out panelsand cable trays to distribute the fiber to the racks 310. Thisarrangement reduces the total number of penetrations into the pressurecontrolled vessel 110 reducing the likelihood of leaks in the vessel.

Some embodiments of the pressure controlled vessel 110 include sensorsto ensure safe operation. These sensors may include, but are not limitedto, temperature sensors, fluid level sensors, pressure sensors 180,gaseous partial pressure sensors, position sensors, electrical sensors,microphones, and/or cameras to ensure and/or automate operations of thesystem.

In one example embodiment, temperature sensors may include, but notlimited to, sensors for measuring the temperature of the gaseous phasewithin the pressure controlled vessel 110, sensors for measuring thetemperature of the liquid phase within the pressure controlled vessel,sensors for measuring the temperature water and/or other process fluids,and/or sensors for measuring the temperature of the other componentsincluding the computer components 170. In some embodiments,thermocouples, thermistors, and/or silicone sensors may be utilized tomeasure the temperature of computer components. In some embodiments, thesystem may rely on information provided by the components themselves andretrieved or monitored through the use of a generally acceptedcommunications protocol, such as a device provided API or otherprogrammatic interface, such JSON via HTTPT or SNMP, to determine theequipment temperature.

Some embodiments may include various life safety features to ensure thesafety of users. These features may include, but are not limited to,automatic electromagnetic locking mechanisms, fail safe systems, fireand/or smoke detection and/or suppression systems, ventilation systems,and/or back up lighting. In certain embodiments, these features may beincorporated as part of a comprehensive platform.

Certain embodiments include an automatic vapor detection based leakdetection system to ensure that any loss of fluid in the pressurecontrolled vessel is rapidly detected. These systems may includepressure sensors 180 within the pressure controlled vessel 110 whichmonitors the pressure in order to ensure there are no substantial leaksand/or gas sensors positioned on the exterior of the pressure controlledvessel which detect the presence of any dielectric vapor which may haveleaked out of the pressure controlled vessel.

The particular design, arrangement, and/or layout of an embodiment ofthe disclosed system may be adjusted based on the conditions in which itis deployed. In some embodiments, the size, material, internal systems,component mounting and configuration options, interfaces between thepressure controlled vessel 110, the computer components 170, and powersystems may all be adjusted based on the conditions in which the systemis utilized.

The Rack System

FIGS. 8A-C show an example embodiment of a rack system 310 (or rack310). The rack 310 may serve as an intermediary between the electricaland communication systems installed within a pressure controlled vessel110 and the computing equipment 170 to be installed within the rack 310.Computer components 170 can be mounted on the rack 310 in order tocontrol the spacing, orientation, position, and/or configuration of thecomputer components 170 in the pressure controlled vessel 110. In oneexample embodiment, each computer component 170 may be installed in achassis 400 before being mounted on the pressure controlled vessel 110.

The rack 310 may be any physical structure which may be used to mountcomputer components 170 including but not limited to any frame, bracket,support, or other structure. Computer components 170 will be consideredmounted to the rack 310 when they are attached, directly or indirectly,to the rack 310 and held in a substantially stationary position. Someembodiments may include the use of dedicated mechanical guide plates asmounting mechanisms, wiring harnesses attached to bulkhead fittings,and/or through the use of intermediary power supplies and a backplanereceiver 331 to distribute power and signal within the rack.

The particular design of the rack system 310 may be adjusted based uponconditions under which the system is deployed. Some embodiments of therack 310 may include a dedicated switch. In some embodiments, the uplinkinterfaces may be connected via fiber infrastructure and/or the downlinkaccess interfaces may be connected to computing equipment 170 within therack via the backplane receiver 331 interface or any other suitable wayof connecting computing equipment.

In certain embodiments, the rack system 310 may include housing for oneor more intermediary power supplies which may distribute the appropriatevoltages from a power interface to other equipment installed within therack 310. The interfaces to interconnect the power from the distributionsystem to the intermediary power supplies may be incorporated into thedesign of the rack 310 to allow it to be removed and/or replaced with analternative rack configuration by disconnecting the interfaces betweenthe various rack, power, and communication systems.

FIG. 8A shows a top view of the rack 310. In this example embodiment,the rack 310 includes an AC interface 311 and a data interface 312. Therack 310 also includes a pair of power supplies, a power supply 313 anda redundant power supply 314 (or backup power supply). The rack 310 mayalso include rectifiers and a controller. The redundant power supply 314(and/or the rectifiers and controller allow the rack 310 to be quicklyrepaired or even to continue functioning if the primary power supplystops functioning. The rack 310 may optionally include a converter 315.The rack 310 is configured to receive a plurality of chassis 400 andhold the chassis 400 in a substantially stationary position.

In some embodiments, the entire rack 310 may be submerged in dielectricfluid. This may include submerging the rectifier, power connections,and/or data connections in dielectric liquid during operation. In orderto reduce and/or eliminate plastic contamination of the dielectricfluid, in some embodiments, plastic insulation and/or cable shieldingmay be eliminated. In such embodiments, the dielectric fluid may serveto insulate the otherwise exposed cables and/or connections.

FIG. 8B shows a perspective view of the rack 310 including a pluralityof chassis 400. The disclosed configuration of the rack facilitates thehot swappability of the chassis 400. In this example embodiment, therack 310 can include a plurality of AC cables 318 which connect the ACinterface 311 to the power supply 313 and/or the redundant power supply314. The power supply 313 and/or the redundant power supply 314 cangenerate a DC output 320 which can be delivered to the backplanereceiver 331 via a DC cable 321. The rack 310 can also include aplurality of data cables 319 which connect the data interface 312 to thebackplane receiver 331. The backplane receiver 331 may be used to supplydata from the data connections on the bottom of the chassis 400 to adata connection at the top of the rack.

FIG. 8C shows a side view of the rack 310. In some embodiments, theracks 310 provides mechanical stabilization and/or housing for thechassis 400 and its components. Additionally, the racks 310 facilitatethe routing of power and data cables from the top of the racks 310,where they are generally accessible within the vessel, to the bottom ofthe racks 310 where they connect with the chassis 400.

The Chassis and Interface Systems

In one example embodiment, the purpose of the disclosed chassis system400 is to serve as a standardized physical intermediary componentbetween traditional and/or purpose built computing components 170 andthe disclosed rack system 310. In one example embodiment, the purpose ofthe backplane receiver 331 is to provide a slot-in interface between thechassis 400 and the rack 310, allowing for the distribution of power andsignals between the power supplies in the power system and the networkswitches in the communication system with the various computingcomponents 170 installed within the chassis 400.

In some embodiments, the pressure controlled vessel of the presentdisclosure can include at least one rack 310, which can include one ormore servers, e.g., blade servers. Each server can be attached to achassis 400 (also called server case or case). FIGS. 9A-G show anexample embodiment of a chassis 400 for mounting various components 170.The chassis can facilitate installation of the servers on the racks ofthe pressure controlled vessel or their removal from the system. In someembodiments, other electronic components of the pressure controlledvessel can be mounted in a chassis. For example, computer components orhardware such as a motherboard, chip, card, any portion of a GPU or CPUcan be installed in a chassis. As another example, components such as apower supply, a power interface, or a network communication interfacecan be mounted in the chassis.

In one example embodiment, the chassis can serve as a common interfacebetween a component (e.g., server) and the pressure controlled vessel.The chassis can provide a variety of mounting, power, and connectivityfeatures which can be customized based on the nature or design of thecomponent. In other words, various aspects of the chassis can bemodified based on the design specification of the component. As such,the chassis can accommodate almost any model or type of hardware. Forexample, the chassis can facilitate usage of specifically designedhardware or off-the-shelf hardware.

Embodiments of the chassis 400 may include components designed to allowfor the adaptation of existing commercial components, the use of customdesigned components, and/or the use of specialized chassis forparticular applications. Embodiments may include adaption kits forstandard motherboards, and specialty components. In particularembodiments, such components include Gigabyte motherboards with NVidiaGPUs and/or Supermicro motherboards with Intel CPUs.

FIG. 9A shows a chassis 400 for mounting a server on a rack according toan example embodiment. In this example embodiment, the chassis 400 canbe a rectangular box including a back wall 410 and two sidewalls 420.The back wall 410 can include a plurality of holes 411 to facilitatecirculation of fluid within the chassis 400. The chassis 400 can includea guide rail 421 on each sidewall 420.

FIG. 9B shows a plurality of components inside of the chassis 400according to an example embodiment. In this example embodiment, the backwall 410 is removed. As such, FIG. 9B shows a server 430 including apower supply module 431, a GPU module 432, a CPU module 433 and aninterface card 434. In one example embodiment, the components inside thechassis 400 can comprise the components used in a blade server, e.g., aCPU module 433 and a GPU module 432. Additionally, the components insidethe chassis 400 can include other components which are traditionally notincluded in a server, e.g., power supply module 431 or interface card434. Because the chassis 400 does not need the traditional air coolingequipment, the chassis 400 does not include any fans or heat sinks inthe chassis. As such, the chassis has a very thin profile relative tothe computing power of the chassis.

FIG. 9C shows a schematic drawing of the components within a chassis. Inthis exemplary embodiment, a server motherboard 445, a plurality ofpower supply modules 431, and an interface card 434 are mounted on achassis 400. Storage devices and/or other peripheral components may alsobe mounted to the chassis 400 along with a backplane interface 330and/or a power module and communication system module.

In one example, mounting interfaces can be added to or removed from thechassis so that a piece of hardware is fixed to the chassis. On aninternal surface of the chassis 400, provisions can be made which allowfor components (e.g., motherboard, GPU, CPU, interface card and otherrelated components) to be mounted to the chassis. These provisions arethe mounting interfaces. The specific arrangement of the chassis system400 may depend on the equipment and/or components that will be attachedto the chassis 400 and/or rack. Some embodiments of the chassis 400 mayfeature an interchangeable mounting plate that can be used for equipmentattachments. A set of standard attachment plates may be used for commonor frequently used components.

The styles and form factors of power and network interface moduleswithin the chassis system 400 may be adjusted based on the demands andrequirements of certain components and/or user specified equipment. Inone example, a power subsystem of the chassis can be modified to addressthe needs of a particular component. In another example, the size of thechassis can be designed to accommodate a piece of hardware of any size.In yet another example, the chassis can offer different networkingoptions depending the network connection card installed in the chassis.Because of these features and other features of the chassis, the chassiscan accommodate a variety of components. As a result, assembly orremoval of these components of the pressure controlled vessel can becomesimplified, and thus, automated. For example, a chassis can include ablade server and a robot can easily install or remove the chassis from arack of the pressure controlled vessel. As such, the robot can removeand replace a blade server without any human interaction, therebyminimizing human exposure to the dielectric fluid.

In one example embodiment, the chassis can include a microcontrollerwhich can be in communication with a management system of the pressurecontrolled vessel. The microcontroller can receive sensor data fromvarious sensors placed in or outside of the chassis. For example, thechassis can include a sensor for detecting whether the chassis isproperly placed in a rack. A server is properly placed in a rack if theserver can make a connection with the rack. The sensor can determinewhether the chassis is properly placed in a rack. As such, the sensorcan transmit data to the microcontroller, and using the data, themicrocontroller can provide a signal to the management system indicatingwhether or not the chassis is properly placed in the rack.

In one embodiment, the microcontroller can be coupled to a switch whichcan power on or off the component mounted in the chassis. Themicrocontroller can receive a power on or off signal from the managementsystem, and in response to receiving the signal, the microcontroller cantransmit a signal to the switch to power on or off the component, e.g.,server. In one example embodiment, the microcontroller can receiveoperational data from the server and the microcontroller can relay thisdata to the management system. Operational data are key performanceindicators of a server and can indicate its performance. Operationaldata can include the speed of the compute, degradation in the compute,power consumption, temperature of the circuits and bandwidth of thesystem.

In one example embodiment, the microcontroller can monitor, manage andcontrol the electrical and communications facilities of a blade server.For example, indicators such as electrical current (i.e., amperage) andvoltage are monitored to make sure that the system can protect itself,e.g., there is no provision of over-current or under-current.

In one example embodiment, the chassis can include a structure which canenable a robot to grab and remove the chassis. For example, the chassiscan be the shape of a rectangular box having front, back and sidewalls.The chassis can also include a top wall and a bottom wall. The top wallof the chassis can include a plate which can make a coupling with arobotic arm. Using the plate, the robotic arm can grab the plate forunloading and other handling operations.

In one example embodiment, the chassis can include mechanical guiderails and positioning pins to ensure proper alignment and insertion ofthe chassis in a rack. Mechanical guide rails may be placed on thesidewalls of the chassis.

In one example embodiment, the chassis can include various features topromote fluid flow. For example, the chassis can be the shape of arectangular box having front, back and sidewalls. The chassis can alsoinclude a top wall and a bottom wall. In this example, at least one ofthe walls of the chassis can include fluid flow holes throughout thewall. For example, the back wall can include a plurality of holes whichcan facilitate fluid flow in and out of the chassis when the chassis isimmersed in a liquid bath.

In one example embodiment, a chassis can include apertures to ensurethat all fluid within the chassis drains when the chassis is removedfrom a liquid bath. For example, a rack can be located in a liquid bathto cool the computer components held by the rack. In order to remove aserver, a robot can grab the plate of a chassis and lift the chassis outof the rack (thereby removing the chassis from the liquid bath). Whenthe chassis is removed from the liquid bath, certain amount of fluid canremain within the chassis. The chassis can include notches or drains atthe bottom wall of the chassis to ensure that fluid can escape even ifthe pressure controlled vessel is not perfectly level. The notches ordrains can be at the corners of the bottom wall.

In one example embodiment, the chassis can include a power interfaceand/or communication interface. The interfaces can electrically couple acomponent mounted within the chassis to the rack and/or the pressurecontrolled vessel. The power interface and/or communication interfacecan be placed at the backplane. For example, a server mounted within thechassis can be connected, via various wires and cables, to the interfaceof the chassis. When the chassis is placed within the rack, theinterface can be electrically coupled to another interface connected tothe rack (i.e., backplane receiver) and/or the pressure controlledvessel. The electrical coupling between the two interfaces (e.g., thebackplane and the backplane receiver) can provide power to the serverand connect the server to a communication network within or outside ofthe pressure controlled vessel. The coupling between the two interfacescan be made automatically during the mechanical insertion of the chassisin the rack. Similarly, removal of the chassis from the rack candisconnect the server from rack and/or the pressure controlled vessel.

In some embodiments, providing standardized interconnectivity throughthe backplane interface 330 and communication system interfaces mayminimize the possibility for misconnection of data interfaces and reducethe need for connectivity troubleshooting.

In certain embodiments, the chassis 400 will include a set of standardpower and network interfaces. The network interfaces may be in the formof Cat6A or Cat7 compatible RJ45 interfaces for connection to 1G or 10GEthernet interfaces on equipment motherboards. In such embodiments, thepower interface may include a set of standard Molex style connectors forthe connection of standard motherboards and/or peripheral components.

In one example embodiment, the pressure controlled vessel can include aninternal database for storing information about the components installedwithin the system. The internal database can be a repository ofcomponents installed on the pressure controlled vessel. For example, theinternal database can store the make and model of every server and powersupply installed within the system. As the components of the system areexchanged or replaced, e.g., by a robot, the management system can keeptrack of the changes and update the information stored on the internaldatabase. The pressure controlled vessel can also be connected to anexternal database via a network.

In one example embodiment, each chassis can be associated with a uniqueserial number, e.g., displayed as a barcode on the chassis. When acomponent is placed within a chassis, the specification of the component(or the component's make and model) can be stored in the externaldatabase in association with the unique serial number. Subsequently,when the chassis is installed in the pressure controlled vessel, thepressure controlled vessel can look up the component by searching theexternal database for the unique serial number. For example, a roboticarm can scan the barcode on the chassis and the management system cansearch the external database using the barcode. The management systemcan update the internal database using the information obtained from theexternal database. Similarly, when a chassis is removed from thepressure controlled vessel, the robotic arm can scan the barcodeassociated with the chassis, and the management system can update theinternal database to indicate that the component mounted in the chassisis not installed in the system anymore.

In one example embodiment, a chassis can include an RFID tag. Therobotic arm of the pressure controlled vessel can include a scannerwhich can emit radio frequency to detect the RFID tag. When the roboticarm is handling a chassis, the robotic arm can scan the RDIF tag andprovide the unique serial number to the management system to update theinternal database.

In one example embodiment, the chassis can include an identificationplate which can contain a user specified asset identification number.This asset identification number can be associated stored in associationwith the component mounted within the chassis. In some embodiments, theidentification plate can be a chip configured to store the assetidentification number.

In one example embodiment, a chassis can include a pump to enhance flowof fluid within the chassis. In order to maximize heat exchange betweena component within a chassis and the liquid bath, the chassis caninclude a pump which can circulate the fluid within the chassis andaround the component. The pump can draw the fluid from various conduitsspread around the chassis and propel the fluid outside of the chassis orvice versa.

In one example embodiment, a chassis can include various conduits aroundthe chassis for drying the chassis and the component mounted therein.When a chassis is pulled from the liquid bath, some amount of liquid mayremain within the chassis or components therein. The chassis can includevarious conduits which can guide a flow of gas within the chassis oraround the component to facilitate drying of the chassis and components.In one example embodiment, the pressure controlled vessel can expose thechassis to a flow of gas before delivering the chassis to a user. Forexample, the chassis can include an input pipe for receiving the flow ofgas and the pressure controlled vessel provide the flow of gas throughthe input pipe.

FIG. 9D shows a bottom wall 415 of the chassis 400 according to anexample embodiment. In this example embodiment, the bottom wall 415 caninclude a power interface 416 and a communication interface 417. FIG. 9Dalso shows the guide rails 421 on sidewalls 420 of the chassis 400.

FIG. 9E shows a top wall 425 of the chassis 400 according to an exampleembodiment. In this example embodiment, the top wall 425 can include aplate 426 and a pair of handles 427. A robotic arm can pick up thechassis 400 using the plate 426.

FIG. 9F shows a sidewall 420 of the chassis 400 according to an exampleembodiment. In this example embodiment, the sidewall 420 can include aguide rail 421. FIG. 9F also shows the back wall 410, the handles 427,and the power interface 416.

FIG. 9G shows an exploded view of a bottom drain hole 450 of the chassis400 according to an example embodiment. In this example embodiment, thebottom drain holes 450 can be placed on the corner of the bottom wall415, sidewall 420, and the back wall 410.

FIGS. 10A-F show an example embodiment of a pressure controlled vessel500. In particular, FIG. 10A shows an exemplary embodiment of a vessel500, e.g., a 600 KW skid. The exemplary embodiment includes a modularskid. The vessel 500 may include a plurality of forklift tubes 514,which facilitate movement and transfer of the vessel 500 to a desiredlocation. The vessel 500 may receive a power and communication input 511and process water from process water pipes 512 with minimal penetrationsthrough the vessel itself. These connections may be positioned on thetop of the vessel in order to facilitate close packing of the modularvessel in a data center. In some embodiments, the connections may beplaced on the front and/or side of the vessel in order to accommodatevertical stacking of multiple modular vessels within a data center. Insome embodiments, a vessel may comprise a vertical spacer to facilitatevessels being stacked vertically on top of each other. The verticalspace may create additional space for connections, air flow, and/orinsulation between vessels. By vertically stacking vessels,extraordinary power density may be achieved on a square foot basis. Insome embodiments, the vessel 500 may include a power and communicationbox configured to receive the input 511 and distribute power and networkconnectivity throughout the vessel 500. The vessel 515 may include asealing lid 515, which may facilitate addition and/or removal ofcomponents from the vessel 500.

FIG. 10B shows another view of the vessel 500. In some embodiments, aninventory of replacement components may be stored within the vessel 500so that components may be replaced using a robotic system within thevessel without opening the vessel. The robotic system may be operatedusing the gantry motors 516. In such embodiments, when a componentbreaks or needs repair, a replacement component is installed into thesystem and the broken or otherwise removed component may be stored in acassette until the cassette is full. At that point, the cassettecontaining removed components may be removed from the vessel and a freshcassette with new replacement components may be inserted into the vesselfor future use. In some embodiments, the disclosed vessel is about 15feet long, about 7 feet wide, and about 10 feet high. In someembodiments, the disclosed system may provide for 600 KW of computingpower to be achieved in about 150 square feet.

In some embodiments, the vessel 500 may also include one or more bellowstank 517. The bellows tank 517 may be used to regulate pressure withinthe vessel. When the disclosed computing and/or cooling system isinitially activated, the expanding dielectric fluid may be directed tothe bellows tank so that it is not lost to the environment and/or toavoid pressure building up within the vessel. In some embodiments, thebellows tank 517 may be large enough to hold about two times the liquiddielectric fluid within the vessel.

FIG. 10C shows a section view of the vessel 500. The lower portion ofthe vessel 500 may contain a rack 310 and/or chassis 400 which containcomputing components. Above the rack is a condenser coil 132 which coolsand condenses any dielectric vapor. Power may be distributed within thevessel using a power bus bar 518. This allows power to be distributed tothe individual computing components in a hot-swappable manner. The powerbus bar 518 allows the vessel to receive external power using only oneor a small number of penetrations through the vessel. This designsimplifies installation and operation of the vessel system. In someembodiments, each power bus bar may serve 600 amps for the powersupplies to five racks. In such embodiments, there may be two sets ofbus bars, one on each side of the vessel. In some embodiments, the busbars do not include plastic insulation. Plastic may be regarded as acontaminant of some dielectric fluids and may be generally avoided insome embodiments.

In some embodiments, the vessel 500 may include a desiccant 519. In someembodiments, dielectric vapor may be removed from the head space of thevessel 500 and condensed in a manner that allows any non-condensableconstituents to be removed from the dielectric fluid. Water will notcondense under the same conditions as many dielectric fluids. As such,this system may be used to remove water contamination from thedielectric fluid.

In some embodiments, the vessel 500 may include a fluid filter 520, afluid pipe 521 and a fluid pump 522. In some embodiments, the dielectricfluid can be added to the vessel in a manner that causes liquiddielectric fluid to spill out of the rack 310 and into the sump area523. Fluid may then be filtered, using the fluid filter 520, and pumped,using the fluid pump 522 and fluid pipe 521, to the far side of thevessel and into the bath portion 528. This system circulates freshfiltered dielectric fluid through the vessel, and thus, the dielectricfluid can be reused to cool the computing components.

FIG. 10D illustrates a section view of the vessel 500. In thisembodiment, a level of liquid dielectric fluid may be maintained at afluid level 524 above the height of the rack 310 and/or the computingcomponents therein. As a result, the rack 310 and/or the computingcomponents may be immersed in the dielectric fluid. Above the fluidlevel 524 may be saturated dielectric vapor, e.g., up to a halfway level525. In some embodiments, the saturated dielectric vapor is maintainedup to the halfway level 525, which may be at about half the height ofthe condenser coil 132. Above the saturated vapor is a headspace whichmay, in some embodiments, contain a less dense dielectric vapor.

The Communication System

Embodiments of the disclosed communication system are designed toprovide a standardized layer 1 through 3 connectivity and managementinterface for the equipment within or associated with the disclosedsuper structure 210, pressure controlled vessel 110, and/or computingsystem.

In some embodiments, a series of MTP interfaces provide the ability tobring multiple high density multimode fiber connections into thepressure controlled vessel 110. Once contained in the pressurecontrolled vessel 110, the fiber connections may be broken down toindividual switch level connections using a set of dedicated break outcables, break out interfaces, patch panels and/or distribution patchpanels to the racks 310.

Some embodiments of the disclosed system may include dedicated fiberpatch panel interface ports at each rack 310 to allow for connection tothe switches system installed therein via a short patch panel. In otherembodiments, there may be a dedicated patch panel, or set of patchpanels, running from each switch system to the MTP distributioninterface.

In some embodiments, the interface between the switch system and thechassis 400 may be via the backplane interface 330 and/or via some othermechanism which may or may not include the use of a backplane connector.In some embodiments, there may be no intermediary rack level switchsystem. Such embodiments may use a set of centralized switches withinthe pressure controlled vessel 110 to connect to various computingequipment located therein.

The typical interface between the switch system and the chassis 400 maybe accomplished using a patch panel attached to the rack 310 and wiredto the backplane system 330 with patch cables connecting the ports onthe patch panel with an appropriate port on the switch system.

In some embodiments, there will be a small (6U) rack rail areacontaining a patch panel interconnecting the communication systemcabinet with the MTP interfaces on each pressure controlled vessel 110,and centralized communication system distribution switches which serveto interconnect the switch system with each other and/or the outsideworld. In such embodiments, the end-user or customer may choose toeither install their own routing gear within this space and provideexternal connectivity thereto to serve as the connection between thedisclosed computing system and the outside world, or run fiberconnectivity between the pressure controlled vessel 110 or superstructure 210 and an existing network environment.

The access, communication, and/or networking components utilized withinan embodiment of the communication system environment may be standardequipment or may be user specified. The rack 310 and backplane interface330 systems may include the ability to replace the switch system locatedwithin each rack 310 by removing the existing switch, replacing it withany standard switch (such as a 1U switch), and re-wiring the desiredinterfaces to the backplane network interface panel.

In certain embodiments, products which are designed to interfacedirectly with the backplane system 330 may be utilized. Such productsmay utilize a chassis 400 patch panel system and/or a direct electricalinterface designed specifically to interconnect switch ports via aspecialized purpose built internetworking interface, via a commerciallyused protocol or via specification for the design of a network levelinterconnectivity interface.

In some embodiments, the connectivity between each blade or chassis andthe switch may contain multiple interfaces. One interface may be astandard switchport which may be the standard port which is available ona commercially available switch. A common interface can be 1GBASE-T or10GBASE-T which makes use of Cat6 or Cat7 twisted pair copperconnections between the switch and the host device. Another interfacemay be a switch-to-backplane intermediary device which may consist ofeither a patch panel with standard patch cables going from the standardswitchports to the front side of the patch panel and a set of hard wiredconnections from the back side of that patch panel to the signalinterfaces of the signal backplane. Alternatively, this could consist ofspecialized cables and/or standard RJ45 interfaces, which go from theswitchport to the board to establish connectivity between the standardswitchports and the backplane. Yet another interface may be an interfacesystem signal backplane which distributes the signal pathways from thestandard switchports along a printed circuit board (PCB). One or more ofthe signal pathways may be terminated at a connector on the PCB to whichthe signal backplane interface will connect. Yet another interface maybe a chassis signal backplane interface. This may be a connector locatedon the chassis itself which mates up to the connector on the interfacesystem signal backplane. It serves as the interface between theinterface system signal backplane with the chassis itself. Yet anotherinterface may be the chassis network interface. This may be a standardpatch interface allowing for the connection of a patch cable from thechassis network interface to the RJ45 interface on the server which isattached to the Chassis.

The Robotic Systems

In some embodiments of the disclosed system, a potential method ofaddressing the need for hot swapability of the components within thepressure controlled vessel 110. The need for the ability to remotelyremove and replace failed components 170 may be addressed throughrobotics.

A particular embodiment of the disclosed combination of systems mayinclude an internal robotic arm 230 and/or external robotic arm 240.Some embodiments, such as those for cryptocurrency applications and/orcertain high performance computing environments, may not require hotswapability of components. In other hyper scale GPU and CPUenvironments, this may be a fundamental requirement. Embodiments of thedisclosed robotic system allow for replacement of a chassis and/or othercomputer component without interruption of any other components. In someembodiments, failed cards and/or components may be automatically and/orprogrammatically replaced and/or stored. This allows for short andmid-term, fully remote and autonomous operation of embodiments of thedisclosed systems.

The internal robotic arm 230 mechanism is located within the pressurecontrolled vessel 110 environment. As shown in FIGS. 7A-D, in anexemplary embodiment, when a card or component is not operatingproperly, a removal sequence may be initiated. When a removal sequenceis initiated, the internal arm 230 will remove the appropriate computercomponent 170 and/or associated chassis 400 from the rack 310, move itto an airlock 220 located within the pressure controlled vessel 110, andsignal completion of the removal sequence. Once this sequence has beencompleted, the inner airlock door 222 will close, the airlock pressurewill be equalized with that of the outside atmosphere, and an exteriorairlock door 224 will open. Once the exterior door 224 has been opened,the external robotic arm 240 will remove the chassis 400 from theairlock 220 and place it into an empty storage slot.

In some embodiments, the airlock 220 will be purged with nitrogen and/oranother inert and/or non-condensable gas, before the airlock 220 isopened to the exterior environment. In such embodiments, this has theeffect of reducing or eliminating the loss of dielectric vapor when theairlock is opened and closed. In certain embodiments, the airlock willbe fit with one-way valves, on the interior portion, exterior portion,or both. In an embodiment with one-way valves on both the interior andexterior portion of the airlock, purging the airlock will prevent crosscontamination of the exterior environment into the interior atmosphereof the pressure controlled vessel 110 and also prevent loss ofdielectric vapor.

When a card or component replacement sequence is initiated, the externalrobot arm 240 will remove a replacement component and/or chassis 400from a storage slot and place the component into the airlock 220. Oncecompleted, the outer airlock door 224 will close, the airlock pressurewill be equalized with that of the inside of the pressure controlledvessel 110, and the inner door 222 will open. Once the inner door 222has been opened, the internal robotic arm 230 will remove the chassis400 from the airlock 220 and insert it into the appropriate rack 310.

When coupled with the remotely accessible management system, theinternal and external robotic arms 230 and 240 allow for the remoteoperation and management of a datacenter environment. This may reducethe need for human operators to remain available and reduce costs and/ordowntime. In some embodiment, the external robot arm 240 is mounted on amovable base, thereby allowing a single external robotic arm system toserve multiple embodiments of the disclosed computing system.

When integrated with custom developed workflow management systems andvirtualization technologies, the disclosed robotic systems allow for thedevelopment of completely autonomous, self-healing datacenter solutionswhich can provide maximum levels of system reliability.

In some embodiments, an asset tag with a unique human and/or machinereadable serial number and/or production batch code may be included oneach computer component, and/or chassis. In these embodiments, the assettag may be a unique serial number. The tag may contain a printed barcodeor QR code and allow for automatic part identification by embodiments ofthe disclosed robotic system. The tag code may also be used inconnection with a management software system to provide detailedcomponent information regarding inventory management and automationsystems. The tag and any associated adhesive or other components arepreferably made of a material that is compatible with the dielectricfluid. The tag is preferably located on the chassis in a spot that isreadable when the chassis is inserted into the rack. In someembodiments, secondary or additional tags may be located on other areaof the chassis to assist in identification of the components and/orinventory management.

Embodiments of the disclosed robotics system allow for any individualchassis to be temporarily removed and replaced, a process called“re-seating”. This is useful when during troubleshooting it isdetermined that a hard power cycle of the component is desired. A reseataccomplishes this by disconnecting all power, waiting a moment, andreconnecting it.

Some embodiments allow for an individual card and/or chassis to beremoved from the pressure controlled vessel through an airlock. In someembodiments, the robotics system will remove a chassis from its slot inthe rack, move it to an airlock, and signal the completion of this taskto allow for the airlock to be opened and the card and/or chassis to beremoved. Some embodiments allow for a replacement component and/orchassis to be placed into a particular rack slot through the sameairlock used for removal. In some embodiments, the robotic system willremove the chassis from the airlock, place it into the appropriate rackslot, and signal its completion of this task.

Robot on the Inside Systems

Embodiments of the disclosed system may contain a “robot inside”robotics system. In such embodiments, the pressure controlled vessel maybe expanded in order to accommodate a robotic arm operating within thevessel. The vessel may also be arranged to accommodate the movement ortransfer of computer components and/or chassis above the racks whichcontain operating computer components. It will be appreciated that apressure controlled vessel may also be referred to as the tank, pod,and/or vacuum chamber. Alternatively, it will be appreciated thatcertain components of the pressure controlled vessel may be referred toas the tank or pod.

FIG. 10E depicts an embodiment of the disclosed system with a gantryrobot 526 configured to remove, replace, and/or install computingcomponents, e.g., chassis 400 of the rack 310. In some embodiments, thegantry robot 526 may also be configured to remove, replace, and/orinstall DC rectifiers and/or other components of the power distributionsystem. It will be appreciated that some embodiments of the disclosedcomputer components and power distribution components may be designed tobe hot swappable and may include handles or other features whichfacilitate handling by the gantry robot 526. In some embodiments, thegantry robot 526 is arranged to travel in both x and y directions andmay drop down in the z direction in order to remove and/or installreplacement components. In some embodiments, the gantry robot 526comprises a gripping tool to grab the chassis 400 and/or power supplies,e.g., the gripping tool can grab the plate 426.

FIG. 10E illustrates a top section view of an exemplary embodiment ofthe disclosed tank. In some embodiments, an array of racks 310 may bepopulated with chassis 400 and/or computing boards. In some embodiments,each chassis 400 may utilize about 6 KW of power and each rack 310 maycontain 10 chassis. Accordingly, in embodiments which contain 10 of suchracks 310, the vessel may utilize about 600 KW of power for computingpurposes. In some embodiments, an additional rack 310 and/or magazine527 of chassis 400 and DC power rectifiers may be stored in the vessel500 to be used as replacement components and/or to provide a space tostore components that have been removed from the vessel 500.

Robot on the Outside Systems

FIGS. 12A-E show another embodiment of the vessel. In particular, FIG.12A depicts an embodiment of the vessel 700 in which the gantry robot526 is exterior of a tank 710 which houses the chassis 400 and/orcomputing components. In this embodiment, the tank 710 may be smallerbut will need to be opened more often for the external gantry robot 526to access the chassis 400 and/or power supplies inside the tank 710.Also, replacement equipment may be stored and/or housed within a modularenclosure such as the storage 716, which is outside of the tank 710. Insome embodiments, the tank 710 may have multiple doors 711, therebylimiting the exposure of the interior of the tank 710 when a single door711 is opened for the purpose of removing, installing and/or replacing acomponent or chassis 400. In such embodiments, replacement componentsmay be stored outside of the tank 710 in order to avoid opening the tankunnecessarily.

Additionally, the vessel 700 may include one or more transformers 712,electrical distribution panels 713, process water pipes 512 andelectrical chase 714. The vessel 700 may also include a programmablelogic controller (PLC) cabinet 715, which monitors and control thestatus of various equipment within the vessel 700. The transformers 712,electrical distribution panels 713, process water pipes 512, electricalchase 714 and PLC cabinet 715 may be located outside of the tank 710.

FIG. 12B illustrates a section view of the vessel 700 with the tank 710being accessible to the external gantry robot 526. In this exampleembodiment, the condenser coil 132, racks 310 and bellows 717 arelocated in the tank 710. FIG. 12C illustrates a side view of the vessel700 with the tank 710 with an external gantry robot and multiple doors711. In this example embodiment, the tank 710 includes a fluid pump forremoving fluid form a sump area and sending the fluid to a fluid filter520 through a fluid pipe 521. The vessel 700 also includes a magazine718 for storage of replacement equipment. In this example embodiment,the magazine 718 is located outside of the tank 710. In someembodiments, spacers and/or ballast blocks 160 may be used in order toreduce the total volume of liquid dielectric fluid in the tank 710.

FIG. 12D illustrates a rack 310 according to an example embodiment. Insome embodiments, a redundant power supply 314 may be positioned on theopposite side of the rack 310 as opposed to adjacent to the primarypower supply 313. Additionally, power and/or data cables 318 and 319 maybe routed in alternative configurations in order to accommodate thespecific requirements of a particular deployment. In this exampleembodiment, a backplane receiver 331 is located under the rack 310.

FIG. 12E shows an example hinging door 711 that may be used in somealternative embodiments of the disclosed tank 710. In some embodiments,sliding doors, as opposed to hinged doors may be utilized in order toreduce or avoid inducing currents in the dielectric vapor. Slowlysliding a door open will likely disturb the dielectric vapor less thanswinging open a hinged door and causing mixing currents.

The Management System

The management system is a web based interface between the user of thedisclosed computing system and the computing system itself. Embodimentsof the management system provide an operational view of the computingsystem and allow for the monitoring and management of the variouscomponents, including monitoring and managing the pressure controlledvessels 110, the robotic systems, the communication system, the powersystem, and/or all other systems and components. In one exampleembodiment, the management system may be implemented in the PLC cabinet715 of FIG. 12A. In another example embodiment, the management systemmay be implemented in the power and communication box 513 of FIG. 10A.in each embodiment, the power management system can be implemented as asoftware program on a control device or other suitable device, e.g., acomputer.

In certain embodiments, a set of simple network management protocolaccessible data points may be made available to users of the managementsystem to allow for monitoring of key operational parameters via thirdparty monitoring systems. Full operational logs may be maintained andcharts may be provided for user review of operational condition data.

Regular maintenance of system components may be scheduled and maintainedvia the management system. The user may be provided with regularreminders of routine maintenance and be able to acknowledge those asbeing performed within the interface. This data may all be maintained aspart of the operational log information for historical operationalreview.

In some embodiments, operational functionality may also be exposed viaan API interface to allow for the remote programmatic monitoring andmanagement of the computing system and associated components. A full setof operational monitoring and alerting functionality may be included toallow for the notification of operators in the event of any issues.

A centralized server version or a hosted cloud based management versionof the management system may be utilized by customers with multiplepressure controlled vessel computing systems. This provides the operatorwith a single programmatically and user accessible interface for themanagement of a fleet of pressure controlled vessel computing systems.

In some embodiments, software based interface modules allow forinteroperation with the computing platform and third party managementutilities, such as Microsoft System Center and VMWare VCenter. The userand API interfaces provided by the management system may allow completeinteroperation with the disclosed robotic systems, allowing for completeremote and programmatic autonomous operation and administration of thedisclosed computing platform.

In some embodiments, control systems allow for adjustment and control ofoperations including temperature, pressure, flow rate, and/or powermanagement. In some embodiments, a user authentication system allowsmultiple unique users to authenticate to the system. Some embodimentsinclude role based and/or element based permission systems. In suchembodiments, an administrator will be able to configure multiple roleswith which users may be associated and/or apply specific permissions toindividual users outside of their role allocations.

Some embodiments incorporate video management in order to provide userswith the ability to record and retrieve video feeds from cameras whichmay be located within a vessel and/or super structure. In someembodiments, the cameras may acquire visual data which may be analyzedby a processor. In such embodiments, the processor may utilize computervision techniques in order to control operations of the vessel,robotics, and/or super structure systems in response to the acquiredvisual data.

In some embodiments, the control system and software may be configuredto generate reports regarding the operations and status of the overallsystem and/or the individual subsystems and/or components of thedisclosed computing platform.

Exemplary Combined System Embodiments

It will be understood that the disclosed systems may be utilizedindividually or in combination. There are multiple embodiments of thecombined computing system which may be tailored to various use cases.

One exemplary embodiment is the Crypto Series. This is an ultra-highdensity implementation of the disclosed technology utilizing purposebuilt computing hardware, racks 310 with guide plates and wiringharnesses designed for that hardware, a modified implementation of thecommunication system 360 architecture, and a 1 MW pressure controlledvessel 110 and power distribution system. The typical user of thisembodiment are those who wish to perform cryptocurrency mining or otherultra-high power density processes using customized computingcomponents, or manufacturers of computing components who wish to developa full scope two-phase liquid immersion cooling system into which theywill incorporate their own hardware.

Another exemplary embodiment is the GPU Series. This is a high densityGPU supercomputing implementation of the disclosed technology. Thisimplementation will make use of custom made chassis 400, rack 310 andbackplane interface 330 technology designed to incorporate motherboardsfrom Gigabyte and GPUs from NVidia using the NVidia NVLink technology tofacilitate ultra-high speed GPU to GPU communications. The typical usersof this technology include general purpose parallel processingapplications which can make use of the GPU based computing and memorycapabilities, including graphics rendering, particle simulation andgeneral research activities.

Yet another exemplary embodiment is the CPU Series. This is a highdensity CPU computing implementation of the disclosed technology. Thisimplementation will make use of high end Supermicro based motherboards,Intel Xeon CPUs, high speed network interfaces, high speed memory, andsolid state storage devices for local storage. The typical user of thistechnology includes datacenter, enterprise, and cloud/VPS hostingproviders and service providers who utilizing high performance computingfor either their own internal applications or for those which theyprovide to third party customers and other organizations.

Still another exemplary embodiment includes the Edge Series. This is ascaled down implementation of the disclosed computing system which isdesigned specifically for remote/field deployments or within oralongside traditional business and data center environments. Theembodiment is focused on a secure, weatherized environment with fullremote monitoring and management capabilities. The target users of thistechnology would be operators of field deployed and distributedtechnologies, such as network operators and other organizations withdistributed field infrastructure, and operators of existing facilitieswishing to augment their computing capabilities with minimalmodification to existing facilities or structures. This system mayincorporate various enhancements to the external structure to simplifythe connection of utility service, including electrical, water andnetwork connectivity to the system.

Self-Contained Embodiments

Some disclosed embodiments do not rely on an external source of water.Such embodiments may comprise a closed-loop chiller for cooling water oranother fluid which may be circulated through a condenser as describedabove. The use of a closed-loop chiller, rather than an external sourceof cooling water allows for embodiments which are substantiallyself-contained.

FIG. 13 shows an example self-contained vessel 750. The exemplaryembodiment of FIG. 13 utilizes a skid-mounted closed-loop chiller 719for cooling water or another liquid to be used in a condenser within thepod or immersion tank 710. By utilizing a closed-loop chiller, the needfor an external source of cooling water may be eliminated resulting in aself-contained data center solution which may require only an externalsource of power and a network connection in order to be fullyoperational. The vessel 750 may also include bellows 717, door 711,gantry robot 526, electrical distribution panel 713, PLC cabinet 715 andmagazine 718.

In some embodiments, the closed-loop chiller 719 may be a skid-mountedclosed loop chiller which is enclosed within an outer housing of amodular pressure controlled vessel. In such embodiments, heat istransferred from computer components to a dielectric liquid within thetank 710. This process converts the dielectric liquid to a dielectricvapor as discussed herein. The dielectric vapor rises within the tank710 and is cooled by a condenser, thereby converting the dielectricvapor back into a dielectric liquid. The heat transferred from thedielectric vapor to the condenser is then transferred from the condenserto a refrigerant or condensation fluid within the condenser and then toa closed-loop chiller 719. In some embodiments, the chiller 719 removesheat from the refrigerant or condensation fluid using vapor-compression,a compressor, an evaporator, a heat-exchanger or other closed-loopmethod of cooling the refrigerant or condensation fluid. The heat fromthe refrigerant or condensation fluid is ultimately dispersed via aircooling. In some embodiments, this results in a self-contained, modular,air-cooled, two-phase liquid immersion computing system. The air coolingof some self-contained embodiments is surprising as the field ofimmersion cooling has commonly taught against air cooling, especiallyair cooling of a self-contained device.

Some disclosed embodiments may be provided in a form factor with aspace-saving footprint. An exemplary embodiment comprises a single rackcontaining ten blades or servers immersed in a dielectric liquid asdescribed above. In some embodiments, each server may draw about 6 kW ofpower. Accordingly, some exemplary embodiments provide about 60 kW ofcomputer power in a small footprint.

The exemplary embodiment illustrated in FIG. 13 is contained within afoot print that is about 4 feet 2 inches deep, about 8 feet 8.5 incheswide, and about 8 feet 8 inches tall. This exemplary embodimentcomprises about 60 kW of computer power as well as the other operationalcomponents and systems and is contained in an area of about 36.3 squarefeet. It will be appreciated that operational components of a vessel mayinclude, but are not limited to, a tank or pod containing a dielectricfluid, condenser, power supply, and data connection for the computercomponents. The vessel may also comprise sensors, control equipment, apower cabinet, a bellows 717, a vacuum system, fluid filter, purgesystem, and/or other components. Some self-contained embodiments maycomprise an outer housing. In some embodiments, the outer housing mayenclose the vessel, provide structural support, be skid mountable, beventilated, be weather and/or water resistant, and/or be decorative. Insome embodiments, the outer housing of a self-contained vessel maycomprise a radiator coil, fan grate, thermal transfer components, and/orair-cooled components to facilitate the use of a closed-loop chiller.

In some embodiments, a self-contained computing system provides at leastabout 1.5 kW of computing power per square foot of area, at least about1.6 kW per square foot, at least about 1.65 kW per square foot, at leastabout 1.8 kW per square foot, at least about 2.0 kW per square foot, orat least about 3.0 kW per square foot. In some embodiments, aself-contained computing system provides at most about 1.5 kW ofcomputing power per square foot of area, at most about 1.6 kW per squarefoot, at most about 1.65 kW per square foot, at most about 1.8 kW persquare foot, at most about 2.0 kW per square foot, or at most about 3.0kW per square foot. It will be appreciated that the height of aself-contained system may be adjusted, thereby allowing for more or lesscomputing power to be provided within a given footprint.

It will be appreciated that the dimensions, components, arrangement, andconfiguration of the disclosed exemplary embodiments may be modified,added to and/or subtracted from to produce a variety of potentialembodiments in a variety of form factors.

In some embodiments, a self-contained computing system may comprise arobotic system, such as, for example, a gantry robot 526, configured toremove, replace, and/or install blade servers, power supplies, or othercomponents, e.g., chassis 400. A self-contained system may compriseeither a “robot on the inside” or a “robot on the outside” system. Inembodiments with smaller footprints, a smaller magazine 718 ofreplacement components may be used. In some embodiments, the magazine718 of replacement components may be attached to the exterior of thetank 710 as shown in FIG. 13. In some embodiments, tank 710, rack,computer components, power supplies, replacement magazine 718, andgantry robot 526 may be arranged such that the gantry robot 526 mayremove, replace, and/or install components while traveling insubstantially only one direction. If the various components are arrangedsubstantially linearly, the gantry robot 526 may be able to travel alonga single axis in order to remove, replace, and/or install the desiredcomponents without traveling in a second direction. It will beappreciated, that the gantry robot 526 may be able to lift and lowercomponents in addition to traveling in a single linear direction.

Utilizing a compact form factor, such as the embodiment illustrated inFIG. 13 allows the self-contained 2PLIC system to be easily transported.The incorporation of a closed-loop chiller 719 allows for two-phaseliquid immersion cooling system to be utilized in remote conditionswhich may not have access to a practical source of chilled water.Additionally, the elimination of a need for external cooling watercreates a self-contained computing system which, in some embodiments,requires only two external connections, a power supply and a dataconnection.

In some embodiments, the computing system may be contained within anouter housing as shown in FIG. 14. In some embodiments, the componentsschematically identified in FIG. 13 and/or disclosed herein may becontained within the outer housing. In some embodiments, the volume ofthe outer housing may be adapted based on the anticipated coolingrequirements, configuration of a closed-loop chiller and/or theenvironment in which the self-contained computing system is expected tobe deployed.

The disclosed self-contained, self-healing, compact form factorembodiments may be used as a stand-alone solution to provide significantcomputing capabilities to almost any location or environment. In someapplications, multiple compact computing systems may be positioned neareach other and/or linked together to create a cluster. In someembodiments, the outer housing is arranged to allow maintenance and/orservice operations to be performed while accessing only one or two sidesof the outer housing. This arrangement allows the individualself-contained computing systems to be positioned with reduced orminimal distance between each self-contained system.

In one example embodiment, clusters of four exemplary self-containedcomputing systems may be positioned strategically to allow for about 240kW of self-contained computer power in about a 140 square footfootprint. In some embodiments, these units may be in power and/or datacommunication with each other, thereby allowing for the operation of amulti-unit cluster with only a single external power connection and asingle data connection. In some embodiments, a data center may beestablished using multiple compact computing systems or multipleclusters of such computing systems.

Some disclosed embodiments and/or computing systems disclosed herein maybe utilized in modern data centers and/or climate controlledenvironments, however some embodiments of the disclosed self-containedcomputing system may be deployed in remote locations and/or harshenvironmental conditions. In some embodiments, the outer housing may beweatherized, waterproof, and/or otherwise arranged to tolerate exposureto harsh environments for an extended period of time. Some disclosedembodiments allow for the rapid deployment of significant computingresources to remote or challenging locations. Some self-containedembodiments may be arranged to be operational in substantially anylocation with access to a power supply and data connection. In someembodiments, an uninterruptable power supply and/or generator may beoperably connected to the computing system to provide more reliable orconsistent access to electrical power.

Some disclosed self-contained embodiments are designed to be stackable.Some stackable embodiments may be designed with a reduced height.Certain embodiments may be about 5′5″ tall, 5′6″ deep and 9′ wide. Themay result in about 60 kW of computer power in a 42 square footfoot-print. Such units may be vertically stacked to provide 120 kW ofcomputer power in the same 42 square foot foot-print.

Embodiments of the disclosed computing system may be stacked andmultiple stacks may be positioned adjacent to each other. This reducesthe need for isle space between individual computing systems, therebyallowing for an overall higher power density within a data center.

Some embodiments may be designed to be fully operable and maintainablewith access to only one side of self-contained computing system. Suchembodiments may be advantageous as they facilitate positioning theself-contained systems in very close proximity to each other.Additionally, in some self-contained embodiments, the entire immersiontank may be removed and/or replaced while accessing only one side of thedevice. In certain embodiments, the tank may be individually modularand/or skid mounted.

In some embodiments, a self-contained computing system may be arrangedvertically to utilize an even smaller footprint. A vertically designedembodiment of the disclosed system may provide about 60 kW of computingpower in about a 22.9 square foot foot-print. As with some otherdisclosed embodiments, some vertically oriented self-contained computingsystems may be positioned in close proximity to one another. Also asdiscussed with some other embodiments, some vertically orientedself-contained computing systems may be operated and maintained withaccess only to one side of the device. In some embodiments, the entiretank may be removed from the outer housing and replaced. Thisarrangement allows for the rapid replacement of multiple blade serversand/or other computing components.

Mobile Embodiments

Self-contained computing systems which do not require an external sourceof cooling water allow for novel computing applications. In someembodiments, power may be provided to the system using a generatorthereby removing the need to connect the system to a source of externaland/or stationary power. In some embodiments, the system may rely onwireless data communication.

In particular self-contained embodiments which do not rely on astationary power source or wired data communication, a fully mobilecomputing system may be implemented. Disclosed embodiments includevehicle mounted, self-contained computing systems which may be used toprovide significant computing power in nearly any environment. In someembodiments, a truck-mounted, wireless computing system may be drivenwithin wireless communication range of an existing or temporary networkand provide a significant amount of computing power with substantiallyno setup or installation time.

Natural Water Embodiments

In some embodiments, a computing system may be arranged to be utilizedon a boat, ship, oil-rig, floating platform, or other vessel orstructure which is located in close proximity to a body of water. Insuch embodiments, the condenser, used to convert dielectric vapor backto a dielectric liquid as discussed herein, may be cooled using waterfrom the body of water. In one exemplary embodiment, a modular computingsystem may comprise a water intake, water output, and pump or impeller.The pump and/or impeller may cause water to flow from the body of water,through the condenser, and then back into the body of water. Someembodiments may comprise filters and/or process components designed toprotect condenser, piping, and other computing system components frompotential sources of contamination in the body of water. In someembodiments, the condenser and other components are arranged to endureextended contact with brackish or salty water such as, for example,ocean water.

Horizontal Magazine Swap

In some embodiments, a magazine of replacement components may be storedon the outside of a tank and within an outer housing of a computingsystem. Replacement components, such as, for example, chassis, servers,blades, and/or power components, may be removed from the magazine andused to replace components within the tank. The magazine may be on aplatform which is configured to extend out of the outer housing of thecomputing system in order to allow components from the magazine to bereplaced.

In one non-limiting example, when a blade server within the tankmalfunctions, a robotic arm may be used to extract the non-workingcomponent from the tank and move the non-working component to a storageslot of a magazine. The robotic arm may then remove a functioning bladeserver from the magazine and install it where the non-working server waspreviously installed, thereby replacing the non-working server with anew working server.

Over time, the magazine will accumulate non-working components which maybe replaced with new working components in order for the robotic systemto continue long term operations. In some embodiments, the magazine ison a platform which may extend to the outside of the outer housing,thereby allowing an operator to access the magazine. In someembodiments, the platform is configured to rotate the magazine from asubstantially vertical position to a substantially horizontal positionin order to allow components to slide in or out of the magazine.

In some embodiments, an adjustable height cart may be used to move,load, and/or receive components so that a human operator is not requiredto lift or support the weight of the components while removing orreplacing components from the magazine. It will be appreciated that amagazine that is configured to rotate to a substantially horizontalposition may also facilitate loading of functioning components into themagazine as well as removing non-functioning components.

FIGS. 15A-D show an example magazine 810 located on a platform 820capable of extending out of the vessel. In FIG. 15A, the magazine 810may be connected to a platform including a rotating member 821, asupporting member 822 and a rail 823. In some embodiments, thesupporting member 822 may be connected to the rail 823 that allows thesupporting member 822 to move while supporting the weight of themagazine 810 and any servers or other components stored within themagazine. In the example embodiment of the FIG. 15A, the platform 820 isin an extended position.

As shown in FIG. 15B, during normal operations, the supporting member822 may be retracted with respect to the outer housing of the computingsystem. The magazine 810 may be stored above the rail 823 during normaloperations. In some embodiments, the weight of the magazine 823 issupported by the supporting member 822 and rail 823 regardless of theposition of the supporting member 822 on the rail 823.

In some embodiments, computer components, such as, for example, servers,utilized with disclosed embodiments may be denser and/or heavier thantraditional computer components. In some embodiments, due to theincreased cooling capabilities of disclosed embodiments, a blade servermay weigh at least as much as about 50 lbs, or at least as much as about60 lbs, or at least as much as about 70 lbs, or at least as much asabout 80 lbs, or at least as much as about 90 lbs, or at least as muchas about 100 lbs. In some embodiments, a blade server may weigh at mostas much as about 50 lbs, or at most as much as about 60 lbs, or at mostas much as about 70 lbs, or at most as much as about 80 lbs, or at mostas much as about 90 lbs, or at most as much as about 100 lbs. As shownin FIG. 15B, a magazine 810 can hold a plurality of chassis 400 or bladeservers where an individual blade server may weigh about 73 lbs. Whenthe magazine is loaded with three such servers, the combined weight ofthe magazine 810 and servers may be about 395 lbs.

In some embodiments, the servers used are blade servers mounted on achassis. The server and/or chassis may contain a backplane system tofacilitate the installation and removal of the servers in the computingsystem. In some embodiments, the servers may be immersion servers whichdo not include a fan or other air cooling devices. In some embodiments,an individual server board may comprise 16 GPU's and be configured todraw about 6 KW of power. In some embodiments, the servers are 1.5Uservers. Some disclosed servers may be 1 Otto Immersion Unit (OIU)servers. Such servers are 1.5U tall and configured for liquid immersioncooling. In some embodiments, a single tank within the computing systemmay be configured to operate ten 1 OIU servers and about 60 KW of powerwhen all ten severs are operating at substantially full power. In someembodiments, the computing system may comprise one or two of such tanks.In some embodiments, the computing system may comprise multiple tankssuch as, for example, ten such tanks.

In some embodiments, when a magazine is extracted from the computingsystem, as shown in FIG. 15A, the supporting member moves along the railfrom a storage position and is cantilevered outside of the outer housingof the computing system.

As also shown in FIGS. 15C-D, the magazine may be pulled out orotherwise slide along the rails and be cantilevered outside of thecomputing system. In some embodiments, as shown in FIGS. 15C-D, amagazine removal tool may be used to remove the entire magazine and thecomponents contained within the magazine. In such embodiments, themagazine removal tool may be used to lift the magazine off of thesupporting member and slide rails in order to transport the magazine.

In some embodiments, once the magazine has been moved to the exterior ofthe computing system, the platform may rotate the magazine to asubstantially horizontal position. The servers contained within themagazine may then slide out of the magazine.

FIGS. 15A-D illustrate an exemplary series of steps for removing aserver from a magazine according to an exemplary embodiment. In theexemplary embodiment, the magazine may be attached to a linear guiderail system behind an access door. As shown in FIGS. 15C-D, the magazinemay be pulled out and cantilevered outside the computing system. Themagazine may be pulled out manually or be moved out of the computingsystem using a motorized or automated system. As shown in FIG. 15D, themagazine may be rotated about 90 degrees to orient the servers and/orother components contained in the magazine in a substantially horizontalposition. Once in a substantially horizontal position, the serversand/or other components can slide out of the magazine and onto a cart orother tool configured to receive the servers and/or other components. Asshown in FIG. 15C, a scissor-lift cart may be adjusted to a convenientheight to receive the servers or other components. An adjustable heightcart with a rolling surface may be used to allow the servers to betransferred from the magazine onto the cart without requiring a humanoperator to support the weight of the servers. As shown in FIG. 15D,once a server is slid onto a cart with a gliding or rolling surface, theserver or other component may be transported elsewhere to be replaced orserviced. It will be appreciated that new components may be loaded intothe magazine using substantially the same steps in the reverse order.

In some alternative embodiments, the magazine may be supported on arotating and extendable arm without a rail. In such embodiments, themagazine may be stored in a substantially vertical position within theouter housing of the computing system during normal operations. Once itis determined that the components within the magazine should bereplaced, the magazine may be extended outside of the outer housingusing the extendable arm. Once the magazine is extended beyond the outerhousing, the magazine may be rotated from a substantially verticalposition to a substantially horizontal position to allow the componentsstored within the magazine to be horizontally removed from the magazine.

Bellows

In some embodiments, a bellows and/or vapor collection system may beutilized. Before some disclosed embodiments are initially activated thedielectric fluid, computer components such as servers, and other systemcomponents may be at thermal equilibrium. Once a computing system isactivated, the computer components, such as servers may begin togenerate heat that maybe dissipated into the dielectric fluid. Thisprocess causes some of the dielectric fluid to shift from a liquid stateto a vapor state. As the temperature of the fluid increases, a greaterproportion of the dielectric fluid may shift to a vapor state. In aclosed system, the increasing volume of dielectric vapor may result inincreased pressure within the system. In some embodiments, the tankcontaining the dielectric fluid may be in fluid and/or vaporcommunication with a recovery system.

FIG. 16 shows a vapor recovery system 900 according to an exemplaryembodiment. The recovery system 900 is connected to the tank 710containing dielectric vapor. Dielectric vapor may flow from the tank 710through piping to one or more bellows 905. In some embodiments, thevapor recovery system 900 comprises an expanding and collapsing bellows905 configured to receive the dielectric vapor, thereby reducing oreliminating any pressure build-up in the tank 710. When the system coolsor a portion of the dielectric vapor is condensed to dielectric liquid,the bellows may collapse or contract to substantially maintain apressure equilibrium within the tank 710.

In some embodiments, the vapor recovery system 900 comprises a valve 912configured to allow ambient air into the vapor recovery system. In suchembodiments, the dielectric vapor may be mixed with ambient air. Mixingdielectric vapor with ambient air may reduce the temperature of thedielectric vapor. In some embodiments, the mixed air/vapor may bedirected through a carbon bed 911. The carbon media within the carbonbed 911 may be configured to attract the dielectric vapor while allowingthe ambient air to pass through the carbon media and be vented from thesystem 900, e.g., through an outlet valve 913. In this manner, theheated dielectric vapor may be cooled and captured by the carbon media.

Upon operating for a sufficient period of time, embodiments of thecomputing system will reach a steady thermal state based on the powercapacity of the computing components being utilized. If more or lesscomputing power is utilized, more or less dielectric fluid may beshifted to dielectric vapor. This may cause the bellows 905 to inflateand/or deflate in response to the heat dissipated into the dielectricfluid.

In some embodiments, the bellows 905 may comprise one or more pouches.Each pouch may comprise a metal foil and polymer laminated construction.The bellows pouches may be connected to the vapor recovery system pipingand to each other in series or in parallel. In some embodiments, thetotal volume of the expanded bellows pouches may be at least about 15%of the liquid fluid volume of the tank. In some embodiments, the totalvolume of the expanded bellows pouches may be at least about 20%, or atleast about 23%, or at least about 25%, of the liquid fluid volume ofthe tank or greater. In some embodiments, the total volume of theexpanded bellows pouches may be at most about 40%, or at most about 30%,or at least about 25%, of the liquid fluid volume of the tank or less.

In some embodiments, when the computing system has substantially reachedthermal stabilization, the vapor recovery system 900 may be closed tocooling ambient air and a valve allowing the air to be exhausted out ofthe system may be closed. In some embodiments, the carbon bed may beconfigured to be opened to only the tank and bellows using valves. Insome embodiments, a desorption heater configured to communicate heat tothe carbon media may be activated to raise the temperature of the carbonmedia. As the carbon media temperature increases, any dielectric fluidpreviously captured by the carbon media may be driven off of the carbonand back into the tank where it can be condensed back to dielectricfluid as previously described.

In some embodiments, when a computing system is powered at less than aprevious steady state, the portion of dielectric fluid in the vaporstate may be reduced and, in some embodiments, the bellows may contractto accommodate the reduction in dielectric vapor. In some embodiments, avalve that allows ambient air into the bellows may be opened in order toallow air into the bellows and further reduce any pressure differential.In some embodiments, nitrogen, rather than ambient air, may be used toreduce a pressure differential and also avoid introducing any potentialcontamination from ambient air.

In some embodiments, the bellows and/or vapor recovery system can becompletely, or substantially, passive. In some embodiments, the bellowsand/or vapor recovery system can be powered and/or automated based onsensor data from temperature, pressure, and/or power sensors positionedthroughout the computing system.

In some embodiments, the computing system with vapor recovery system isemission free even if the system is not a closed system. In someembodiments, ambient air or nitrogen may be introduced into the systemand exhausted out of the system without releasing any or substantiallyany dielectric fluid into the surrounding atmosphere.

Exemplary Embodiments

Disclosed embodiments allow for increased density of computer componentsand/or computing power. In some embodiments comprising two-phase liquidimmersion cooled computer components 170 within a pressure controlledvessel 110, components may be separated from each other by less thanabout 1″ or less than about 0.7 inches, or less than about 0.5 inches.In some embodiments, individual components may be separated by more thanabout 0.3 inches, or more than about 0.5 inches, or more than about 0.7inches, or more than about 1 inch, or more than about 1.5 inches.

Some disclosed embodiments allow improved power utilizationeffectiveness (PUE) as compared to a traditional data center. Usingdisclosed embodiments allows for reduced energy usage for coolingcomputer components 170, thereby reducing the total energy usage of adata center and bringing the PUE closer to 1.0. Some embodiments relateto data centers comprising two-phase liquid immersion cooled computercomponents within a pressure controlled vessel 110 wherein the datacenter has an PUE of less than about 1.15, or less than about 1.10, orless than about 1.08, or less than about 1.05. Some embodiments relateto data centers comprising two-phase liquid immersion cooled computercomponents within a pressure controlled vessel 110 wherein the datacenter has an PUE of more than about 1.05, or more than about 1.06, ormore than about 1.08, or more than about 1.10.

In some embodiments a thermally conductive condensable dielectric fluidis provided to be used in a two-phase liquid immersion cooling system.Computer components are operated under less than ambient atmosphericpressure which reduces the temperature at which the dielectric fluidvaporizes, thereby maintaining the liquid phase of the dielectric fluidat a lower temperature as compared to standard atmospheric pressure. Thecomputer components generate heat as they operate. The generated heat istransferred to the dielectric liquid in contact with the computercomponents, causing the dielectric liquid to vaporize into a gas. Thegaseous dielectric fluid may be condensed using a condenser. Ambienttemperature, or chilled process water is passed through the condenser.When the gaseous dielectric fluid is cooled by the condenser, itcondenses back to a liquid phase and falls back into a bath of liquiddielectric fluid.

Some disclosed embodiments relate to high density data centers.Traditional data centers include about 1 megawatt (MW) of computingpower dispersed over about 10,000 square feet. High end data centers mayinclude about 1 MW of computing power dispersed over about 6,000 squarefeet. Disclosed embodiments relate to data centers comprising two-phaseliquid immersion cooled computer components 170 within a pressurecontrolled vessel 110 wherein the data center utilizes about 1 MW ofcomputing power dispersed over about 3,000 square feet, or about 1,500square feet, or about 1,000 square feet, or about 800 square feet, orabout 600 square feet. In some embodiments, multiple pressure controlledvessels containing the disclosed computing system may be arranged inrows and powered by a central power supply. In some embodiments,multiple embodiments of the disclosed computing system may be connectedto each other in series.

Disclosed embodiments comprise liquid immersion cooled computercomponents 170 within a pressure controlled vessel 110, accordingly, thecomponents are insulated from atmospheric contamination by the pressurecontrolled vessel and by being submerged in dielectric liquid 140. Somedisclosed embodiments relate to data centers which operate with minimalair filtration and/or cleaning requirements. In some embodiments, thedata center operates in the absence of HEPA filters or the equivalent,or in the absence of MERV 11 filters or the equivalent, or in theabsence of MERV 8 filters or the equivalent.

Disclosed embodiments comprise liquid immersion cooled computercomponents 170 within a pressure controlled vessel 110, accordingly, thecomponents are not cooled by air of gases. Disclosed embodiments includea data center which operates in the absence of cooling fans and/or anyother similar device for circulating air.

Disclosed embodiments relate to environmentally friendly data centers.In some embodiments, a data center comprise liquid immersion cooledcomputer components 170 within a pressure controlled vessel 110 andconsume little to no water for cooling processes. Some embodimentsutilize a closed circuit dry cooling tower to reduce the temperature ofwater which is circulated through the disclosed condensing structures130 in order to cool the condensing structures 130 and condensedielectric fluid vapor into dielectric fluid liquid. Such embodimentsoperate without significant input or output of water as the closed loop,dry cooling tower does not rely on evaporative cooling or a stream ofwater for cooling operations. Some data center embodiments utilizeand/or discharge less than about 10,000 gallons of water per day, orless than about 1,000 gallons of water per day, or less than about 100gallons of water per day, or less than about 10 gallons of water perday, or 0 gallons of water per day. Some data center embodiments utilizeand/or discharge more than about 100 gallons of water per day, or morethan about 1,000 gallons of water per day, or more than about 10,000gallons of water per day.

Disclosed embodiments relate to a computing system comprising a pressurecontrolled vessel operably connected to a pressure controller and/orsource of vacuum, wherein the pressure controlled vessel has an interiorand an exterior and is configured to contain an atmosphere within theinterior; a volume of thermally conductive, condensable dielectricfluid; a rack for mounting computer components, wherein the rack isarranged such that the computer components are at least partiallysubmerged within the volume of thermally conductive dielectric fluidwhen mounted on the rack; and a condensing structure, wherein the volumeof thermally conductive dielectric fluid, rack, computer components, andcondensing structure are contained within the pressure controlledvessel. Some embodiments relate to a cooling system comprising apressure controlled vessel comprising an interior wherein said vessel isconfigured to be operably connected to a pressure controller to reducethe interior pressure below atmospheric, wherein the pressure controlledvessel is configured to comprise a volume of thermally conductive,condensable dielectric fluid in liquid and gas phase; one or morecomputer components arranged such that the one or more computercomponents may be at least partially submerged within the liquid phaseof a volume of thermally conductive, condensable dielectric fluid; and acondenser for condensing gas phase dielectric fluid to liquid phasedielectric fluid.

In some embodiments, the pressure controlled vessel is mounted within asuper structure, the blade server is configured to be swappable withoutdisruption of the computing system, the pressure controlled vessel isoperably connected to a power supply, source of water, and networkingconnections, the pressure controlled vessel comprises an opening on thetop and a lid configured to sealably close the opening, the lid isconfigured to direct rising vapors from the middle of the pressurecontrolled vessel to the sides of the pressure controlled vessel, thepressure controlled vessel has an interior volume of between about 100cubic feet and about 300 cubic feet, and/or wherein the pressurecontrolled vessel contains a ratio of liquid dielectric fluid to gaseousdielectric fluid of between about 1:3 and about 1:8. Some embodimentsfurther comprise a ballast block, a blade server and a blade serverchassis, a robotic arm and an airlock, wherein the airlock is configuredto allow access to the interior of the pressure controlled vesselwithout significantly disrupting the atmosphere within the pressurecontrolled vessel, and/or a purge system, wherein the purge system isconfigured to remove contaminants from the volume of thermallyconductive dielectric fluid. In some embodiments, the purge system isconfigured to remove a portion of the atmosphere from the pressurecontrolled vessel, condense any dielectric fluid from the atmosphere,and discard any remaining vapors. In some embodiments, the purge systemis configured to condense at least a portion of gaseous dielectricfluid, and discard gaseous contaminants.

Some embodiments relate to a method for cooling computer components, themethod comprising: providing a housing, wherein the housing contains athermally conductive, condensable dielectric fluid and heat-generatingcomputer components, and wherein the housing is configured to withstandat least a slight vacuum; operating computer components, whereinoperating the computer components generates heat and wherein thecomputer components are in contact with the dielectric fluid; andcreating a vacuum within the housing, wherein the pressure within thehousing is at least below about 1 atmosphere. Some embodiments furthercomprise maintaining the vacuum within the housing, wherein the pressurewithin the housing is below about 1 atmosphere while the computercomponents are operating, vaporizing the dielectric fluid from a liquidstate to a gaseous state using the heat generated by the computercomponents and condensing the dielectric fluid from a gaseous state to aliquid state using a condenser, removing fluids which are not readilycondensable from the dielectric fluid. In some embodiments, and/orreplacing a portion of the computer components while the system isoperating. In certain embodiments removing non-condensable fluidscomprises isolating a portion of the gaseous atmosphere from within thehousing, condensing any dielectric fluid from the gaseous atmosphere;returning the condensed dielectric fluid to the housing, and discardingany remaining portion of the gaseous atmosphere and/or the housing isconfigured to generate a convection current.

Some embodiments relate to a method of cooling computer componentscomprising operating computer components at less than ambient pressure,wherein the computer components are in contact with a thermallyconductive, dielectric fluid. Some embodiments further comprisevaporizing the dielectric fluid and condensing the dielectric fluid atless than ambient pressure.

Some embodiments relate to a method for cooling computer components, themethod comprising: providing a thermally conductive, condensabledielectric fluid in a liquid and gas phase; and operating computercomponents at a pressure below ambient atmospheric pressure in thepresence of the thermally conductive, condensable dielectric fluid,wherein the computer components are at least partially in contact withthe thermally conductive, condensable dielectric fluid in liquid phase.Some embodiments further comprise vaporizing the dielectric fluid from aliquid phase to a gas phase using at least a portion of any heatgenerated by the operating computer components; condensing at least aportion of the dielectric fluid from a gas phase to a liquid phase;removing at least a portion of non-readily condensable fluids from thedielectric fluid; and/or replacing at least one or more computercomponents while said computer components are operating.

Some embodiments relate to a method of cooling computer components, themethod comprising: operating computer components at least at 1 psi lessthan ambient pressure, wherein the computer components at leastpartially in contact with a thermally conductive, dielectric fluid, andwherein the boiling point of the dielectric fluid is below about 80° C.Some embodiments further comprise condensing the dielectric fluid atconditions such that the computer components do not exceed about 80° C.

It will be understood that the various disclosed embodiments mayincorporate some or all of the components otherwise described herein.The particular components and the properties thereof may be adjustedbased on the properties of each particular embodiment. Modifications mayinclude the use of higher or lower density power, cooling, and networkconnectivity systems, pressure management systems, vapor managementsystems and selection of particularized equipment and components.

From the foregoing description, one of ordinary skill in the art caneasily ascertain the essential characteristics of this disclosure, andwithout departing from the spirit and scope thereof: can make variouschanges and modifications to adapt the disclosure to various usages andconditions. The embodiments described hereinabove are meant to beillustrative only and should not be taken as limiting of the scope ofthe disclosure.

Process of Immersion Cooling

1. A method comprising:

at least partially submerging a computer component in a thermallyconductive, condensable dielectric fluid, wherein:

-   -   the computer component is mounted in a chassis comprising a        backplane for receiving power from a rack; and    -   the computer component is configured to dissipate heat in the        dielectric fluid when the computer component operates;

condensing, using a condenser, a gas phase of the dielectric fluid to aliquid phase of the dielectric fluid;

wherein the rack is within a tank comprising a pressure controller toreduce or increase an interior pressure of the tank.

2. The method of embodiment 1, wherein the tank has a computing powerdensity of at least 300 W of computing power dispersed over each squarefoot of space.

3. The method of embodiment 1, further comprising removing the chassisfrom the rack using a robot, wherein the robot is located within thetank.

4. The method of embodiment 3, further comprising delivering, using therobot, the chassis to an airlock, wherein the airlock is configured toallow access to an interior of the tank without significantly disruptinga pressure within the tank.

5. The method of embodiment 4, further comprising:

opening an inner door of the airlock;

placing the chassis in the airlock;

closing the inner door of the airlock;

equalizing a pressure of the airlock with an atmospheric pressure; and

opening the outer door of the airlock.

6. The method of embodiment 3, further comprising storing, using therobot, the chassis in a magazine.

7. The method of embodiment 6, wherein the magazine is located on aplatform including a supporting member, a rotating member and a rail.

8. The method of embodiment 3, wherein the robot is a gantry robotconfigured to remove, replace, or install the chassis.

9. The method of embodiment 8, wherein the gantry robot is configured tomove on a horizontal plane and drop down vertically.

10. The device of embodiment 9, wherein the robot is configured toremove, replace, or install a component of a power distribution system.

11. The method of embodiment 10, wherein the robot includes a grippingtool for grabbing the chassis.

12. The method of embodiment 1, wherein the tank is mounted within asuper structure which includes a plurality of tanks.

13. The method of embodiment 1, further comprising removing contaminantsfrom the dielectric fluid.

14. The method of embodiment 1, further comprising removing gaseouscontaminants.

15. The method of embodiment 1, further comprising providing power,network connection and process fluid to the tank.

16. The method of embodiment 1, wherein the tank comprises an opening onthe top and a removable lid.

17. The method of embodiment 1, wherein the tank comprises an interiorvolume of between about 100 cubic feet and about 300 cubic feet.

18. The method of embodiment 1, wherein the chassis does not include aheat sink and a fan.

19. The method of embodiment 1, wherein the chassis includes a bladeserver, a processor, a power supply or an interface card.

20. The device of embodiment 19, wherein the backplane is electricallyconnected to an interface card, which is a Cat6A or a Cat7 compatibleRJ45 interface for connection to a 1G or a 10G Ethernet interface.

Vessel Design and Configurations for Immersion Cooling

1. A device comprising:

a tank configured to hold thermally conductive, condensable dielectricfluid;

a pressure controller to reduce or increase an interior pressure of thetank;

a rack at least partially submerged within the dielectric fluid;

a condenser for condensing a gas phase of the dielectric fluid; and

a robot configured to move a chassis within the rack.

2. The device of embodiment 1, wherein the device comprises a modularskid comprising a plurality of forklift tubes.

3. The device of embodiment 1, wherein the tank has a computing powerdensity of at least 300 W of computing power dispersed over each squarefoot of space.

4. The device of embodiment 1, wherein an exterior of the deviceincludes a power input and a communication input.

5. The device of embodiment 4, wherein:

-   -   the power input and the communication input are electrically        connected to a box; and    -   the box, using a plurality of wires, distributes the power input        and the communication input to the rack.        6. The device of embodiment 5, wherein the rack includes a        backplane receiver configured to distribute power and a        communication signal to the chassis.        7. The device of embodiment 6, wherein the chassis includes a        backplane configured to:

receive the power and the communication signal from the backplanereceiver of the rack; and

distribute the power and the communication signal to a computercomponent within the chassis.

8. The device of embodiment 5, wherein the plurality of wires do notinclude plastic insulation.

9. The device of embodiment 5, wherein the rack includes a transformer.

10. The device of embodiment 1, wherein the device is stackable.

11. The device of embodiment 1, wherein the device comprises a magazinefor storage of replacement components.

12. The device of embodiment 11, wherein the robot is configured toremove the chassis from the rack and place the chassis in the magazine.

13. The device of embodiment 12, wherein the magazine is located on aplatform which includes a rotating member, a supporting member and arail.

14. The device of embodiment 13, wherein the platform is configured toguide the magazine outside of the device.

15. The device of embodiment 1, wherein the device includes a desiccantconfigured to remove water vapor contamination from the device.

16. The device of embodiment 1, further comprising:

a sump area;

a pump; and

a filter;

wherein the pump is configured to remove the dielectric fluid from thesump area and pass the dielectric fluid through the filter beforedelivering the dielectric fluid to a bath portion of the tank.

17. The device of embodiment 1, wherein the dielectric fluid has aboiling point within a range of 20° C. to 100° C.

18. The device of embodiment 1, wherein the dielectric fluid comprises achemical with a formula of, (CF3)2CFCF2OCH3, C4F9OCH3, CF3CF2CF2CF2OCH3,hydrofluoro ethers or methoxy-nonaflurobutane.

19. The device of embodiment 1, further comprising a lock that precludesthe device from operating if any of a lid or a door of the device is notsecured.

20. The device of embodiment 1, further comprising a controllerconfigured to power down the device in the event of an unauthorizedaccess to the lid or the door.

Robotics and Automation for Immersion Cooling

1. A device comprising:

a tank configured to hold a thermally conductive, condensable dielectricfluid;

a pressure controller to reduce or increase an interior pressure of thetank;

a computer component at least partially submerged within the dielectricfluid;

a condenser for condensing a gas phase of the dielectric fluid; and

a robot configured to pick up the computer component.

2. The device of embodiment 1, further comprising an airlock.

3. The device of embodiment 2, wherein the airlock includes an innerdoor and an outer door.

4. The device of embodiment 3, wherein the airlock is configured toreceive an inert gas to purge the gas phase of the dielectric fluidbefore the outer door is opened.

5. The device of embodiment 3, wherein the robot is located outside thetank.

6. The device of embodiment 3, wherein the robot is located within thetank.

7. The device of embodiment 6, wherein the robot is configured to removethe computer component from a rack and deliver the computer component tothe airlock.

8. The device of embodiment 7, wherein the robot is further configuredto:

open the inner door of the airlock;

place the computer component in the airlock;

close the inner door of the airlock;

equalize a pressure of the airlock with an atmospheric pressure; and

open the outer door of the airlock.

9. The device of embodiment 8, further comprising a second robot locatedoutside of the tank.

10. The device of embodiment 9, wherein the second robot is configuredto remove the computer component from the airlock when the outer door isopen.

11. The device of embodiment 9, wherein the second robot is configuredto place the computer component within a storage slot.

12. The device of embodiment 9, wherein the airlock is configured toequalize the pressure of the airlock after the outer door is closed.

13. The device of embodiment 1, wherein the device is configured toreceive instructions from a server located outside of the device.

14. The device of embodiment 1, wherein the computer component islocated within a chassis showing an asset tag.

15. The device of embodiment 14, wherein the robot is configured to scanthe asset tag and relay the asset tag to a management system.

16. The device of embodiment 1, wherein the robot is a gantry robotconfigured to remove, replace, or install the computer component.

17. The device of embodiment 8, wherein the gantry robot is configuredto move horizontally and vertically.

18. The device of embodiment 1, wherein the robot is configured toremove, replace, or install a component of a power distribution system.

19. The device of embodiment 10, wherein the component of the powerdistribution system is a transformer or a power supply.

20. The device of embodiment 1, wherein the robot includes a grippingtool for gripping the computer component.

Ballast Blocks for Immersion Cooling

1. A device comprising:

a tank comprising:

-   -   a bath portion for holding thermally conductive, condensable        dielectric fluid and a computer component; and    -   a shelf portion configured to hold at least one ballast block;

a pressure controller to reduce or increase an interior pressure of thetank;

a condenser for condensing a gas phase of the dielectric fluid; and

a robot configured to pick up the computer component.

2. The device of embodiment 1, wherein a bottom point of the bathportion has a lower height than a height of the shelf portion.

3. The device of embodiment 1, wherein the bath portion is configuredfor the computer component to be at least partially submerged in thedielectric fluid.

4. The device of embodiment 3, wherein the computer component is a bladeserver, a processor, a power supply, or a transformer.

5. The device of embodiment 1, wherein a level of the dielectric fluidis high enough to cover at least part of the shelf portion.

6. The device of embodiment 1, wherein the shelf portion is next to thecondenser.

7. The device of embodiment 6, wherein the shelf portion is configuredto receive condensed dielectric fluid from the condenser.

8. The device of embodiment 1, wherein the ballast block is configuredto occupy a volume of the tank above the shelf to displace thedielectric fluid from the shelf to an area over the bath portion.

9. The device of embodiment 1, wherein the ballast block includes aplurality of riser feet for allowing the dielectric fluid to flowunderneath the ballast block.

10. The device of embodiment 1, wherein the ballast block is not solublein the dielectric fluid.

11. The device of embodiment 1, wherein the ballast block is made from ametal, a rubber, a silicone, or a polymer.

12. The device of embodiment 1, wherein the ballast block is denser thanthe dielectric fluid.

13. The device of embodiment 1, wherein the ballast block has a handle,cut out or a plate for removal or replacement of the ballast block.

14. The device of embodiment 13, wherein the robot is configured to liftthe ballast block using the handle, the cut out or the plate.

15. The device of embodiment 1, wherein the ballast block is configuredto interlock with another ballast block from a top side or a bottom sideof the ballast block.

16. The device of embodiment 15, wherein the interlocking prevents theother ballast block from sliding.

17. The device of embodiment 15, wherein the other ballast block isconfigured to be on the top side or the bottom side of the ballastblock.

18. The device of embodiment 15, wherein the ballast block comprisesrecessed portions on the top side of the ballast block such that a riserfeet of the other ballast block is configured to lock in one of therecessed portions of the ballast block.

19. The device of embodiment 1, wherein the ballast block is configuredto span at least 40% of an entire length of the shelf portion.

20. The device of embodiment 1, wherein the ballast block has an outerdimensions of about 2 feet long, about 8 inches wide and about 1 inchtall.

Server Case for Immersion Cooling

1. A device comprising:

a tank configured to hold thermally conductive, condensable dielectricfluid;

a pressure controller to reduce or increase an interior pressure of thetank;

a chassis at least partially submerged within the dielectric fluid;

a condenser for condensing a gas phase of the dielectric fluid; and

a robot configured to pick up the chassis.

2. The device of embodiment 1, wherein the chassis does not require aheat sink or a fan.

3. The device of embodiment 1, wherein the chassis includes a bladeserver.

4. The device of embodiment 1, wherein the chassis includes a processor,a power supply or an interface card.

5. The device of embodiment 19, wherein the interface card is a Cat6A ora Cat7 compatible RJ45 interface for connection to a 1G or a 10GEthernet interface.

6. The device of embodiment 1, wherein the chassis is removably attachedto a rack.

7. The device of embodiment 6, wherein the chassis includes a backplaneto provide a slot-in interface between the chassis and the rack.

8. The device of embodiment 7, wherein the backplane is configured todistribute power and signals received from the rack within the chassis.

9. The device of embodiment 8, wherein the backplane is configured totransmit power and data to a blade server via a cable.

10. The device of embodiment 1, wherein the chassis is a substantiallyrectangular box comprising a back wall and two sidewalls, wherein theback wall has a plurality of holes to facilitate circulation of thedielectric fluid within the chassis.

11. The device of embodiment 10, wherein the chassis comprises a guiderail on each of the two sidewalls.

12. The device of embodiment 1, wherein the chassis comprises a mountinginterface for holding computer components.

13. The device of embodiment 1, wherein the chassis comprises a plateand the robot is configured to lift the chassis using the plate.

14. The device of embodiment 1, wherein the chassis includes amicrocontroller.

15. The device of embodiment 14, wherein the microcontroller isconfigured to:

receive sensor data from a sensor mounted on the chassis, the sensordata indicating whether the chassis is properly placed in a rack; andtransmit the sensor data to a management system.

16. The device of embodiment 14, wherein the microcontroller isconfigured to:

receive a power signal from a management system; and

transmit the power signal to a switch configured to cutoff the powerwithin the chassis.

17. The device of embodiment 14, wherein the microcontroller isconfigured to: receive operation data from a computer component mountedwithin the chassis; and transmit the operation data to the managementsystem.

18. The device of embodiment 14, wherein the microcontroller isconfigured to control electrical and communication facilities of a bladeserver.

19. The device of embodiment 1, wherein the chassis comprises an RFIDtag.

20. The device of embodiment 19, wherein the robot is configured to scanthe RFID tag and transmit a signal to a management system.

Vapor Management for Immersion Cooling Using Bellows

1. A device comprising:

a tank configured to hold thermally conductive, condensable dielectricfluid and a computer component;

a pressure controller to reduce or increase an interior pressure of thetank;

a vapor management system for condensing a gas phase of the dielectricfluid; and

a robot configured to pick up the computer component.

2. The device of embodiment 1, wherein the vapor management systemincludes a condensing structure within the tank.

3. The device of embodiment 2, wherein the condensing structure includesa thermally conductive tube, a coil or radiator fins.

4. The device of embodiment 2, wherein the condensing structure isconfigured to be coupled to a source of cooling liquid so that thecooling liquid passes through the condensing structure.

5. The device of embodiment 2, wherein the device is configured to chillthe cooling liquid using evaporative cooling or dry cooling towers.

6. The device of embodiment 2, wherein the vapor management systemincludes an incoming pipe and an outgoing pipe.

7. The device of embodiment 6, wherein the incoming pipe is configuredto receive cooling liquid from a chilled cooling liquid source and guidethe cooling liquid through the condensing structure.

8. The device of embodiment 6, wherein the outgoing pipe is configuredto receive cooling liquid from the condensing structure and return thecooling liquid to the chilled cooling liquid source.

9. The device of embodiment 1, wherein the vapor management systemincludes a storage unit for storage of the dielectric fluid.

10. The device of embodiment 9, wherein the vapor management system isconfigured to direct the dielectric fluid in the tank from the storageunit.

11. The device of embodiment 1, wherein the vapor management systemincludes a vapor storage unit for storage of vapor of the dielectricfluid.

12. The device of embodiment 11, wherein the vapor storage unit is abellows.

13. The device of embodiment 12, wherein the bellows are configured toinflate or deflate to maintain the interior pressure of the tank.

14. The device of embodiment 12, wherein the bellows comprises one ormore pouches.

15. The device of embodiment 11, wherein the vapor storage unitcomprises a valve for allowing air into the vapor management system toreduce a temperature of the vapor of the dielectric fluid.

16. The device of embodiment 15, wherein the vapor storage unit isoperably connected to a carbon bed to separate the vapor of thedielectric fluid from air.

17. The device of embodiment 16, wherein the carbon bed comprises adesorption heater configured to heat the carbon bed to raise thetemperature of the carbon bed.

18. The device of embodiment 1, wherein the vapor management systemcomprises a filter.

19. The device of embodiment 17, wherein the filter is configured toremove air and water vapor.

20. The device of embodiment 1, wherein the vapor management system:

comprises an inert gas storage unit; and

is configured to introduce an inert gas from the inert gas storage unitinto the tank during a startup operation or a shutdown operation.

What is claimed is:
 1. A device comprising: a tank configured to holdthermally conductive, condensable dielectric fluid; a pressurecontroller to reduce or increase an interior pressure of the tank; arack at least partially submerged within the dielectric fluid; acondenser for condensing a gas phase of the dielectric fluid; a robotconfigured to move a chassis within the rack; and a sump area; a pump;and a filter; wherein the pump is configured to remove the dielectricfluid from the sump area and pass the dielectric fluid through thefilter before delivering the dielectric fluid to a bath portion of thetank.
 2. The device of claim 1, wherein the device comprises a modularskid comprising a plurality of forklift tubes.
 3. The device of claim 1,wherein an exterior of the device includes a power input and acommunication input.
 4. The device of claim 3, wherein: the power inputand the communication input are electrically connected to a box; and thebox, using a plurality of wires, distributes the power input and thecommunication input to the rack.
 5. The device of claim 4, wherein therack includes a backplane receiver configured to distribute power and acommunication signal to the chassis.
 6. The device of claim 5, whereinthe chassis includes a backplane configured to: receive the power andthe communication signal from the backplane receiver of the rack; anddistribute the power and the communication signal to a computercomponent within the chassis.
 7. The device of claim 4, wherein theplurality of wires do not include plastic insulation.
 8. The device ofclaim 4, wherein the rack includes a transformer.
 9. The device of claim1, wherein the device is stackable.
 10. The device of claim 1, whereinthe device comprises a magazine for storage of replacement components.11. The device of claim 10, wherein the robot is configured to removethe chassis from the rack and place the chassis in the magazine.
 12. Thedevice of claim 11, wherein the magazine is located on a platform whichincludes a rotating member, a supporting member and a rail.
 13. Thedevice of claim 12, wherein the platform is configured to guide themagazine outside of the device.
 14. The device of claim 1, wherein thedevice includes a desiccant configured to remove water vaporcontamination from the device.
 15. The device of claim 1, wherein thedielectric fluid has a boiling point within a range of 20° C. to 100° C.16. The device of claim 1, wherein the dielectric fluid comprises achemical with a formula of, (CF3)2CFCF2OCH3, C4F9OCH3, CF3CF2CF2CF2OCH3,hydrofluoro ethers or methoxy-nonaflurobutane.
 17. The device of claim1, further comprising a lock that precludes the device from operating ifany of a lid or a door of the device is not secured.
 18. The device ofclaim 1, further comprising a controller configured to power down thedevice in the event of an unauthorized access to the lid or the door.